Container Runtime Interface in Kubernetes 1.6

Kubernetes 1.6 was released March 26, 2017.

What’s new in Release 1.6

According to the blog post from Kubernetes, this release focuses on scale and automation. Mirantis has a very good “What’s new in Kubernetes 1.6”. In this article, it listed the following categories of major changes:

• DaemonSet rolling updates
• Kubernetes Federation
• Authentication and access control improvement
• Scheduling changes
• Container Runtime Interface is now the default
• Storage improvements
• Networking Improvements
• Other Changes

“Other changes” is the catchall category for those changes that are also important. For all the changes in release 1.6, check out the release notes on GitHub.

Kubernetes also has a blog post describing release 1.6.

Container Runtime

 Kubernetes is a container orchestration engine. For container to run on the host, it needs to have a container runtime. Back in release 1.0, Kubernetes only support the Docker container runtime – runc. In release 1.3, rkt is added. In release 1.5 the Container Runtime Interface is added to allow Kubernetes to support a wider range of container runtime to integrate with kubelet on a node. The container runtime interface in Kubernetes 1.5 release as alpha and the Docker container runtime remains to be the default. With this interface, for Kubernetes to support a new container runtime, it does not need to be integrating deep in the kubelet source code.

What is Container Runtime Interface?

 In brief, the Container Runtime Interface is an abstraction layer allowing kubelet to interface with any container runtime. Before release 1.5, without this interface, adding container runtime support will have to make coding changes to the kubelet source code.

This diagram explains how the Container Runtime Interface works:

 image source: http://blog.kubernetes.io/2016/12/container-runtime-interface-cri-in-kubernetes.html

Container Runtime Interface interacts with kubelet uses the gRPC protocol. This blog post from Kubernetes has a more detailed description on Container Runtime Interface. Like any open source project GitHub usually has good documentation on the subject.

Container Runtime Interface is turned on as the default behavior in Kubernetes 1.6 even it is still in beta status. Beside runc and rkt, currently these container runtime are in developement to work with CRI:

cri-o: OCI conformant runtimes.
rktlet: the rkt container runtime.
frakti: hypervisor-based container runtimes.
docker CRI shim.

Kubernetes Resources

• Kubernetes blog
• Kubernetes tutorials
• Staratoscale wiki
• Kubernetes the Hardway
• Weaveworks blog: Kubernetes Beginner's Guide

OpenStack is now one more step closer to "enterprise" ready

In the month of July, groups around the world the world are celebrating the 5 year birthday of OpenStack. They are giving out  t-shirts to commemorate this event along with birthday cakes and champagne (each group is celebrating a little bit differently). 

OpenStack started in 2010 with is first release named as Austin.  Eric Wright (@discoposse) had a nice post on OpenStack Kilo release and have a detailed description of some interesting statistics about OpenStack as well as some of OpenStack's latest feature.

OpenStack came a long way in these 5 years and is starting to gain more and more attention as a solution to business needs.  Last OpenStack summit in Vancouver there were over 6,000 attendees.  New features in respond to the market needs were being added in a high pace.  While OpenStack is feature rich and is delivering these features in a fast pace with 2 releases per year, one question there is always one question in mind - 

  “Is OpenStack enterprise ready?"

Essential "enterprise" features

I think the top 2 essential enterprise features are:

1. High Availability
2. Security

High Availability
One important aspect for enterprise is to sustain continuous business operation.  To look at a product if it is enterprise ready we must look at what level of HA it is able to deliver.  Different business have different requirement for the level of HA a product can provide. If we look at that OpenStack web site, it stated that OpenStack is able to offer 99.99% of uptime for the infrastructure. But if we read on, it stated that OpenStack does not guarantee 99.99% availability for individual guest instances.  Well it does not mean that OpenStack is NOT enterprise ready.  At least the OpenStack is guaranteed to have 99.99% of uptime.  If this cannot be achieved, then we can forget about the 99.99% of the individual guest instances.

Vendor specific build-in HA features

There are a few ways that we can take advantage of the existing and proven high availability feature provided by vendors such as:

1. Red Hat Enterprise Linux 7 - Pacemaker
2. Ubuntu 14.04 - High Availability Cluster solutions
3. SUSE Linux Enterprise High Availability
4. VMware vSphere infrastructure 

Kubernetes
Another development in OpenStack is project Magnum where OpenStack interacts with Kubernetes to orchestrate the deployment of groups of containers as a Pod and there is the replication controller in Kubernetes that user can defined the desired state of the Pod in such a way that if container dies, the Replication Controller is going to re-create the Pod defined by the user.

image source: https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAjyzgb5tK336Z93gwLMTxC2iSaolGw3SNof_YQZwRa3TrJYuk8OCVBFBuCatAqWhOWedOV7CNto-pB5XCXTe74XIlBFonwUbD-FRveDGpQZIGAwJbsheJPwWk8pb2RUNYC3aKUOV9hYAB/s1600/k8s+replication+controllers+(1).png

Security

Another important aspect for enterprise is security.  OpenStack has a project called Keystone that handles the security of the infrastructure.  One feature for Keystone is that it can use Active Directory as a backend server and this allows OpenStack to use the existing and widely used Active Directory.  This is helping OpenStack to be enterprise ready.

Interaction within the various element of the OpenStack infrastructure is via RESTful API.  Each entity will have to request a security token for each API call so that the target entity can validate if the initiator of the API call has the proper credential and privilege to request the operation.

OpenStack has a dedicated security team to look at the security of the OpenStack infrastructure.  Each commit to the OpenStack code base has a keyword SecurityImpact that the commiter can use to flag the security team to investigate if that particular commit has any security impact to OpenStack.

With all these features in place, more and more companies are willing to deploy OpenStack in the production environment.  This is a sample of companies that are deploying OpenStack in their production environment (just showing some examples and is not a complete list):

 

image source: http://cloudtweaks.com/wp-content/uploads/2014/11/companies-snapshot.png 

Within the last few days there are 2 new development that helped move OpenStack closer to be "enterprise" ready.

Google becoming a sponsor to OpenStack Foundation

On July 16, Google announced that it is becoming a sponsor to the OpenStack Foundation along with “big” companies such as IBM, Red Hat, Ubuntu or VMware support this open source cloud infrastructure.  I had a blog post on my reaction to Google’s announcement.

The significant of Google sponsoring OpenStack Foundation is twofold.  The first one is that Google is in a way endorsing this technology.  The second one is what its knowledge, expertise and experience on deploying container at scale to make OpenStack able to deploy container as well as virtual machine in it infrastructure.

OpenStack Innovation Center

On July 23, there is another OpenStack announcement that is worth looking into and I believe is as important as the Google announcement. This important announcement is that Rackspace and Intel is collaborating to form an OpenStack Innovation Center.  According to the press release the purpose of this OpenStack Innovation Center is to help accelerate the development of enterprise capabilities and thus driving the adoption of OpenStack in the enterprise production environment.

The agreement between Rackspace and Intel includes:

• OpenStack Innovation Center – The center will create the world’s largest OpenStack development team. It will be comprised of Rackspace and Intel engineers and will be located at Rackspace’s corporate headquarters in San Antonio.
• OpenStack Developer Training – Through this effort, Rackspace and Intel will offer new modules of courseware designed to onboard and increase the number of open source developers actively contributing to the success of the community.
• Joint OpenStack Engineering – Rackspace and Intel will resource OpenStack development, working in collaboration with the OpenStack Enterprise Work Group and community, targeting bug elimination and the development of new enterprise features. The companies will recruit new engineers to participate in OpenStack development.
• Largest OpenStack Developer Cloud– Rackspace and Intel will build and make available to the community two 1,000 node clusters to support advanced, large-scale testing of OpenStack and new features developed through the joint engineering work. The companies anticipate having the clusters available for community use within the next six months.

One good news for me as a software developer on this collaboration between Rackspace and Intel is that there will be a two 1,000-node OpenStack hybrid cloud clusters that will be made available to the OpenStack community to look a issues regarding scaling and performance.

What is next?
Wonder what is in the horizon for more enterprise features in OpenStack.  I had a conversation with a person that is in the OpenStack community and he was saying that there will be a lot more exciting announcements in the upcoming OpenStack Tokyo summit and in the OpenStack Austin summit next year. 

Reference:
"Accelerating the Enterprise Features of OpenStack: Rackspace and Intel Form the OpenStack Innovation Center." Rackspace Hosting. N.p., n.d. Web. 25 July 2015.

Marriage made in heaven - Google + Project Magnum

On July 16, Google announced that it is becoming a sponsor to the OpenStack Foundation.  This announcement immediately generated a lots of excitement and this news is all over the social media.  Immediately lots of blogs and technical news outlets published articles on this announcement.

My immediate reaction was this:

as I express myself on Twitter.  

In this blog post, Product Manager of Google, Craig McLuckie stated that by becoming a sponsor to the OpenStack Foundation, Google is expected to bring valuable knowledge, expertise and deployment experience of container-oriented computing to OpenStack. The blog also stated that Google is seeing 2 emerging trends in the enterprise computing sector:

1. Moving toward the adoption of hybrid cloud (public and private)
2. Moving toward the adoption of container-based computing

Quote directly from the blog post:

"We (Google) will be working over the coming months with the (OpenStack) community to integrate Kubernetes, as well as complementary container technologies, to create a stronger hybrid cloud."

The excitement about Google sponsoring OpenStack Foundation is not just just another "big" company joining IBM to endorse the future of OpenStack but what Google can bring and to enhance what OpenStack can provide for its consumers. Google has been running containers for over 10 years. Google's various applications such as Gmail, web search or map services are running as containers. It is said on the internet that Google launch 2 billions containers in one week.

Why Linux Container?
Linux Container is not a new technology and its origin can trace back to FreeBSD Jails in year 2000. Docker Inc, is the company behind the open source Docker platform. The Docker platform is an orchestration or packaging tool that allows application and their dependencies to run on container technology. Docker makes it easy for developer and sys admin to deploy applications with container technology.

The advantage of Linux container is light weight and fast provision as compare to a virtual machine.  It is also portable

Linux containers is just perfect for DevOps as well as Micro-services architecture.

Linux Container is part of the equation along with DevOps and Micro-services to answer the modern day business requirement – agility. In other words the ability to react to the change of the business environment fast or faster than the competitor.

Both DevOps and Micro-services are big topics that will take up an entire blog post to talk about.

In a nutshell, DevOps is not a technology but a culture and a form of application deployment methodology.  It relies on the continuous feedback model from operation to development so that applications can be deployed in a faster manner.

image source: http://www.appdynamics.com/blog/wp-content/uploads/2013/06/DevOps-infinity-loop2.png

Micro-services is the new and popular way of deploying applications.  Application used to be one monolithic process and is not as agile.  Micro-services breaks the application into smaller processes running together.

 image source: http://www.thoughtworks.com/sites/default/files/assets/MicroservicesFowler.png

DevOps, Linux container and micro-services architecture fits right into each other because they share one common characteristic and that is to break one complex problem into smaller problems which can be resolved easier and faster.

Linux containers are gaining traction in the enterprise IT environment.  In my other blog post, I talked about VMware embracing Docker with Project Bonneville making Docker to run just like a virtual machine in vSphere so that Docker container can take advantage of the matured high availability, security and networking infrastructure that VMware had developed.  

Microsoft is also embracing Docker in its public cloud Azure.

We can see that container is an emerging technology integrating with various enterprise ready technologies independently providing features for consumer of the IT infrastructure or cloud.

What is Kubernetes?

Kubernetes is developed by Google to manage containers in a clustered environment.  Google makes Kubernetes open source and can be found on GitHub here.  

The beauty of open source project is that there is always description of the project.  The  description of Kubernetes on GitHub is:

"Kubernetes is an open source system for managing containerized applications across multiple hosts, providing basic mechanisms for deployment, maintenance, and scaling of applications."

An interesting point to note is that Kubernetes is still considered as pre-production beta status and heavy development is still being done. (Well GitHub said it is in pre-production beta but on July 21 at oscon, Kubernetes 1.0 is released.  Release 1.0 and above is usually production ready).

Kubernetes works off the following concepts:

image source: 13110153-conversion-gate01/95/devnexus-2015-kubernetes-container-engine-17-638.jpg?cb=1426244619

Red Hat also used Kubernetes on their PaaS offering - OpenShift.

I will write up a more detailed blog post on Kubernetes later.  If you cannot wait, take a look at here.

Project Magnum
Before the July 16 Google announcement, OpenStack already has an ongoing project called Magnum - Containers as a Service. 

Project Magnum allows containers be deployed in an OpenStack infrastructure as easy as a virtual machine but in the fashion that container should be deployed.  This opens up a lot more business use case on DevOps and Micro-services for OpenStack.

In Nova, Docker is considered as one of the supported hypervisor type.  The main difference for Magnum over Nova is that Magnum uses container orchestration tool to deploy containers at scale and in a rapid manner.  Deployment characteristic of a container is different than a virtual machine such that container come and go more frequently and containers usually are deployed in group as a cluster and sometimes across multiple hosts.  These containers in the cluster needs interact with each other when they operate together to offer the service of one application to the user as in the Micro-service architecture.

There is not much description about the Magnum on GitHub.  Better description can be found on OpenStack wiki page for Magnum The OpenStack Magnum wiki page even has a 5 minute video on how Magnum works.  This project on OpenStack wiki is described as:

"OpenStack API service making container orchestration engines such as Docker Swarm and Kubernetes available as first class resources in OpenStack"

The main idea is to take advantage of the existing Docker orchestration engine and make it work on an OpenStack infrastructure.

I will write another blog post to describe Magnum in more detail.  For now we will look at Magnum at a very high level.  As a preview, the architecture of Magnum looks like this:

 

image source: https://wiki.openstack.org/wiki/File:Magnum_architecture.png

With this diagram, we can see the general idea on how Magnum works.  Magnum also takes advantage of the OpenStack Heat project.

This slide from the OpenStack Vancouver summit presentation summarize what is on the OpenStack Magnum wiki page:

image source:http://image.slidesharecdn.com/tuckeropenstacksummitvancouverpossibilities-150525221145-lva1-app6891/95/openstack-in-an-ever-expanding-world-of-possibilities-vancouver-2015-summit-25-638.jpg?cb=1432592195

What is ahead of us?
I see that Project Magnum is the reaction to the modern day business requirement and now with Google being able to inject its valuable knowledge, expertise and deployment experiences to the project, Linux container deployment at scale in OpenStack will become mature in due time. It will also integrate better with the other OpenStack projects such as Swift, Keystone or Neutron that more business use cases can be supported.

Google has over 15 years of deploying application at large scale and will certainly able to bring its experience and expertise to the Magnum project.

The most important question is how fast and what features Project Magnum can produce in a timely manner meaning if the market is ready to embrace what is offered. Sometimes the technology is way more advanced than what the market is ready for. I see in the 2000, there is a start-up already trying to offer SDN but at that time the market is not ready for it and that start-up closed down due to lack of funding around 2003.


Reference:
- "Containers + Private Cloud. Google Sponsors OpenStack Foundation." Google Cloud Platform Blog. N.p., n.d. Web. 17 July 2015. 
- "GoogleCloudPlatform/kubernetes." GitHub. N.p., n.d. Web. 18 July 2015. 
- "Magnum." - OpenStack. N.p., n.d. Web. 18 July 2015.     

 

Networking options for Docker

Docker is gaining popularity both in the new and in actual deployment.  Last time I talked about VMware is also embracing Docker technology.

Lets take a look at the different networking options for Docker.

Native Docker Networking

• On startup Docker creates a Ethernet Bridge docker0 on the Linux Kernel
• docker0 creates a virtual subnet on the Docker host
• Docker creates a pair of virtual Ethernet interface on each container
• One of the Ethernet interface is the eth0 in the container
• Another Ethernet interface will have a unique name in the form of veth* (e.g.vethABI3IC) and is bind to docker0
• User can customize docker0
• Advanced Docker networking can be found here

image source: http://www.infrabricks.de/assets/images/docker_network_basics1.png

Flannel

• reference articles An etcd backed overlay network for container, article1 and article2
• Used to be called Rudder
• An etcd backed overlay network for container
• flannel uses etcd for storage of both configuration data and subnet assignments.
• Upon startup, a flannel daemon will retrieve the configuration and a list of subnet already in use
• Select an available subnet (a randomly picked one) and attempt to register it by creating a key in etcd
• Works under Kubernetes
• Resolve the problem of native Docker networking where network address was hidden from the network outside of the Docker host machine
• It can work on any exiting network without changes to that network.

 

image source: https://github.com/coreos/flannel/blob/master/packet-01

Weave

• reference article
• An overlay networking system for Docker container
• Open source back by a commercial company Zettio
• The system consists of two components:
• Weaver - a virtual network switch and router implemented in Go that runs within a Docker container on each networked host.
• Weave - a script that wraps the Docker command line in order to start Weaver, connect containers to the Weave virtual network, and assign IP addresses to them.
• Zettio founder Alexis Richardson said "Weave makes the network fit the application, not the other way round"
• Can be think of as "a giant Ethernet switch to which all the containers are connected"
• Weave network can be encrypted
• Simplies the process of connecting containers together instead of using Open vSwitch

image source: http://www.infoq.com/resource/news/2014/09/zettio_releases_weave/en/resources/1Weave.png

Pipework

• reference article.
• Docker creates a special Linux bridge called docker0 on startup. 
• All containers are automatically connected to this bridge and the IP subnet for all containers is randomly set by Docker. 
• Currently, it is not possible to directly influence the particular IP address of a Docker container.  
• pipework is a shell script which user can use to add another interface to a container with a specified IP address.

SDN
SDN is an hot topic and Docker is another hot topic and what is going to happen when we put SDN and Docker together?

Recently a new startup SocketPlane has an idea of bringing Open vSwitch to the Docket host.  I read an article from SDN Central about SocketPlane:

• To have a container that is going to be able to manage the data path
• Also able to manage either overlays or underlays
• Will focus on network management and network services instead of the network system
• Will also address problems with DevOps with various Configuration Management Tools

When we go to SocketPlane's website you will see "Native to Docker", "Familiar to NetOps" and "Application Friendly".  I think this 3 phases summarize the product direction that this company is heading.  One thing to note is that the founders of this startup are all veterans of the OpenDayLight projects along with former executive from OpsCode/Chef.

I have another post on OpenDayLight previously, please take a look if you are interested. :)

Bring SDN to Docker can open up using a container for NFV or in this case NFD (Network Function Dockerization) <- a new name that I come up with based on the word "Dockerize"

Reference:
"CoreOS." CoreOS Blog ATOM. N.p., n.d. Web. 26 Oct. 2014.
"Coreos/flannel." GitHub. N.p., n.d. Web. 26 Oct. 2014.
"Zettio Releases Weave for Docker Networking." InfoQ. N.p., n.d. Web. 26 Oct. 2014.
"Docker Networking Made Simple or 3 Ways to Connect LXC Containers - Codecentric Blog." Codecentric Blog. N.p., n.d. Web. 26 Oct. 2014.
"OpenDaylight Coders Open Shop on a Docker Startup." SDNCentral. N.p., n.d. Web. 26 Oct. 2014.
"Docker Networking Made Simple or 3 Ways to Connect LXC Containers - Codecentric Blog." Codecentric Blog. N.p., n.d. Web. 26 Oct. 2014.