OpenStack is now one more step closer to "enterprise" ready

In the month of July, groups around the world the world are celebrating the 5 year birthday of OpenStack. They are giving out  t-shirts to commemorate this event along with birthday cakes and champagne (each group is celebrating a little bit differently). 

OpenStack started in 2010 with is first release named as Austin.  Eric Wright (@discoposse) had a nice post on OpenStack Kilo release and have a detailed description of some interesting statistics about OpenStack as well as some of OpenStack's latest feature.

OpenStack came a long way in these 5 years and is starting to gain more and more attention as a solution to business needs.  Last OpenStack summit in Vancouver there were over 6,000 attendees.  New features in respond to the market needs were being added in a high pace.  While OpenStack is feature rich and is delivering these features in a fast pace with 2 releases per year, one question there is always one question in mind - 

  “Is OpenStack enterprise ready?"

Essential "enterprise" features

I think the top 2 essential enterprise features are:

1. High Availability
2. Security

High Availability
One important aspect for enterprise is to sustain continuous business operation.  To look at a product if it is enterprise ready we must look at what level of HA it is able to deliver.  Different business have different requirement for the level of HA a product can provide. If we look at that OpenStack web site, it stated that OpenStack is able to offer 99.99% of uptime for the infrastructure. But if we read on, it stated that OpenStack does not guarantee 99.99% availability for individual guest instances.  Well it does not mean that OpenStack is NOT enterprise ready.  At least the OpenStack is guaranteed to have 99.99% of uptime.  If this cannot be achieved, then we can forget about the 99.99% of the individual guest instances.

Vendor specific build-in HA features

There are a few ways that we can take advantage of the existing and proven high availability feature provided by vendors such as:

1. Red Hat Enterprise Linux 7 - Pacemaker
2. Ubuntu 14.04 - High Availability Cluster solutions
3. SUSE Linux Enterprise High Availability
4. VMware vSphere infrastructure 

Kubernetes
Another development in OpenStack is project Magnum where OpenStack interacts with Kubernetes to orchestrate the deployment of groups of containers as a Pod and there is the replication controller in Kubernetes that user can defined the desired state of the Pod in such a way that if container dies, the Replication Controller is going to re-create the Pod defined by the user.

image source: https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAjyzgb5tK336Z93gwLMTxC2iSaolGw3SNof_YQZwRa3TrJYuk8OCVBFBuCatAqWhOWedOV7CNto-pB5XCXTe74XIlBFonwUbD-FRveDGpQZIGAwJbsheJPwWk8pb2RUNYC3aKUOV9hYAB/s1600/k8s+replication+controllers+(1).png

Security

Another important aspect for enterprise is security.  OpenStack has a project called Keystone that handles the security of the infrastructure.  One feature for Keystone is that it can use Active Directory as a backend server and this allows OpenStack to use the existing and widely used Active Directory.  This is helping OpenStack to be enterprise ready.

Interaction within the various element of the OpenStack infrastructure is via RESTful API.  Each entity will have to request a security token for each API call so that the target entity can validate if the initiator of the API call has the proper credential and privilege to request the operation.

OpenStack has a dedicated security team to look at the security of the OpenStack infrastructure.  Each commit to the OpenStack code base has a keyword SecurityImpact that the commiter can use to flag the security team to investigate if that particular commit has any security impact to OpenStack.

With all these features in place, more and more companies are willing to deploy OpenStack in the production environment.  This is a sample of companies that are deploying OpenStack in their production environment (just showing some examples and is not a complete list):

 

image source: http://cloudtweaks.com/wp-content/uploads/2014/11/companies-snapshot.png 

Within the last few days there are 2 new development that helped move OpenStack closer to be "enterprise" ready.

Google becoming a sponsor to OpenStack Foundation

On July 16, Google announced that it is becoming a sponsor to the OpenStack Foundation along with “big” companies such as IBM, Red Hat, Ubuntu or VMware support this open source cloud infrastructure.  I had a blog post on my reaction to Google’s announcement.

The significant of Google sponsoring OpenStack Foundation is twofold.  The first one is that Google is in a way endorsing this technology.  The second one is what its knowledge, expertise and experience on deploying container at scale to make OpenStack able to deploy container as well as virtual machine in it infrastructure.

OpenStack Innovation Center

On July 23, there is another OpenStack announcement that is worth looking into and I believe is as important as the Google announcement. This important announcement is that Rackspace and Intel is collaborating to form an OpenStack Innovation Center.  According to the press release the purpose of this OpenStack Innovation Center is to help accelerate the development of enterprise capabilities and thus driving the adoption of OpenStack in the enterprise production environment.

The agreement between Rackspace and Intel includes:

• OpenStack Innovation Center – The center will create the world’s largest OpenStack development team. It will be comprised of Rackspace and Intel engineers and will be located at Rackspace’s corporate headquarters in San Antonio.
• OpenStack Developer Training – Through this effort, Rackspace and Intel will offer new modules of courseware designed to onboard and increase the number of open source developers actively contributing to the success of the community.
• Joint OpenStack Engineering – Rackspace and Intel will resource OpenStack development, working in collaboration with the OpenStack Enterprise Work Group and community, targeting bug elimination and the development of new enterprise features. The companies will recruit new engineers to participate in OpenStack development.
• Largest OpenStack Developer Cloud– Rackspace and Intel will build and make available to the community two 1,000 node clusters to support advanced, large-scale testing of OpenStack and new features developed through the joint engineering work. The companies anticipate having the clusters available for community use within the next six months.

One good news for me as a software developer on this collaboration between Rackspace and Intel is that there will be a two 1,000-node OpenStack hybrid cloud clusters that will be made available to the OpenStack community to look a issues regarding scaling and performance.

What is next?
Wonder what is in the horizon for more enterprise features in OpenStack.  I had a conversation with a person that is in the OpenStack community and he was saying that there will be a lot more exciting announcements in the upcoming OpenStack Tokyo summit and in the OpenStack Austin summit next year. 

Reference:
"Accelerating the Enterprise Features of OpenStack: Rackspace and Intel Form the OpenStack Innovation Center." Rackspace Hosting. N.p., n.d. Web. 25 July 2015.

KVM – a free and yet powerful hypervisor

Setting up a home lab had been a struggle for me because of budget.  A few years back after I passed the VCP 4 and 5 (I passed both within a week because I have to take advantage of the free VCP upgrade exam) I was so happy to have my copy of VMware Workstation.  With the VMware Workstation I can run virtual machine on my Windows 7 server (Dell T110 with 16G memory).  I think a few years back VMware Workstation is the best to offer as a hypervisor for the home lab.

VMware is doing a great job not only on it server virtualization product but also has pushed other hypervisor maker to produce better product.  As a software developer I used Unix as a development platform for a long time.  Lately with my involvement with OpenStack I get to play with Ubuntu a lot more.  The biggest step that I have taken is bought for myself a laptop where I can dual boot Windows 8.1 and Ubuntu desktop.  As I have used Ubuntu more and more I have discovered that KVM is a very powerful hypervisor and yet it is free.

 image source: http://www.linux-kvm.org/kvmless/kvmbanner-logo3.png

How does KVM work?

KVM stands for Kernel-based Virtual Machine. KVM is part of the Linux Kernel since version 2.6.20.  

KVM turns the Linux Kernel into a Type 1 hypervisor.  This is an essential characteristics of KVM.  Difference between Type 1 and Type 2 hypervisor can be found here

In Linux term the hypervisor is known as Virtual Machine Monitor (VMM).

Wikipedia has a good picture on how KVM works:

image source: https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine#/media/File:Kernel-based_Virtual_Machine.svg

To understand how KVM works there are 3 key concepts that we need to understand:

1. kvm.ko - a module in the Linux kernel
2. QEMU - short for Quick Emulator 
3. Guest Mode

kvm.ko

kvm.ko is the driver in the Linux kernel that interacts with the guest operating system of the virtual machine that is running on the user space of the host operating system.  This driver consists of 3 files:

• kvm.ko - provides the core virtualization infrastructure
• kvm_intel.ko - specific to the Intel processor
• kvm_amd.ko - specific to the AMD processor

It used to be a loadable module and starting from Linux version 2.6.20, this module is part of the mainline module.

On your Linux host machine, with the modprobe command we can see:

atc@atc-OptiPlex-740:~$ modprobe -l | grep kvm
kernel/arch/x86/kvm/kvm.ko
kernel/arch/x86/kvm/kvm-intel.ko
kernel/arch/x86/kvm/kvm-amd.ko

kvm.ko is responsible for switching the host processor into the "Guest" mode.

QEMU

image source: https://dw1.s81c.com/developerworks/mydeveloperworks/blogs/a2674a1d-a968-4f17-998f-b8b38497c9f7/resource/BLOGS_UPLOADED_IMAGES/Screenshot-2012-07-0615%3A53%3A40.png

The correct term should be qemu-kvm.  There is the regular QEMU (Quick Emulation) that does machine emulation to run guest operating system on a Linux machine.  The regular QEMU is making the host Linux machine a Type-2 hypervisor.  KVM make use of the regular QEMU framework to host the guest operating system and make changes to interact with the kvm.ko module in the kernel of the host Linux machine. To install KVM on a Ubuntu system we have to install the qemu-kvm package.  Since qemu-kvm and QEMU are so close together that a lots of people use QEMU as a generic name for the module that runs the guest operating system. 

Guest Mode

Linux has the concept of user mode and kernel mode.  This blog has good information on these 2 modes.  KVM introduce a third mode - Guest Mode:

This is where the guest operating system can have its own user and kernel mode while running at the user mode of the host Linux system under qemu-kvm.

Guest mode is implemented as a hardeare file - /dev/kvm which acts as an interrupter between the actual hardware of the host system and the VMM. When KVM is install you will see:

atc@atc-OptiPlex-740:/dev$ l -lt /dev/kvm
crw-rw----+ 1 root kvm 10, 232 Jul 10 10:33 /dev/kvm


 

                               image source: http://www.linuxjournal.com/files/linuxjournal.com/linuxjournal/articles/102/10251/10251f1.jpg

For a more detail description of the guest-mode, The Linux Journal has a good article here.

KVM features
It is true that KVM is free and is efficient but what feature does it offer?  Is KVM enterprise ready?

This KVM page has a list of KVM feature but with a commend saying it is not the complete list of features for KVM.  Live migration is one of the list feature and I believe this is a very important, attractive, fundamental  and necessary feature for KVM to be enterprise ready.

When it comes to Linux and enterprise, Red Hat has its RHEV (Red Hat Enterprise Virtualization) and is being used in various data centers.  Canonical Ltd is the company behind Ubuntu is catching up on its "enterprise ready" distribution.

SUSE which is popular in Europe also has it version of "enterprise ready" distribution.

I will in another blog post compare these 3 versions of Linux distribution that has value-add feature for KVM for commercial use.


Management for KVM

This page has a list of ways to manage KVM.  This page is pretty up-to-date because I see Platform9 being listed.  Platform9 is a new startup that ship product to manage OpenStack.

Most management tool be it web based (e.g. oVirt by Red Hat), GUI based (e.g. virt-manager) or command line (e.g. virsh) used libvirt as the interface to manage KVM.

libvirt
I found an excellent article that explain libvirt in detail.  It explains what libvirt is and lists out some major function of libvirt. In the article it also explains how to install and configure libvirt.

IBM blog has a good article on libvirt. Again a picture is worth a thousand words, this diagram explain how libvirt fit into KVM management:

image source: http://www.ibm.com/developerworks/library/l-libvirt/figure3.gif

How to install KVM

There are lots of blog post covering how to install KVM.  I have followed this post to install KVM on my Ubuntu laptop.  One basic prerequisite to run KVM is that the hardware that your Linux is running supports hardware virtualization.  This can easily be find out by typing this on the terminal of a Linux machine.

 egrep -c ‘(svm|vmx)’ /proc/cpuinfo

If it returns a non zero value, the hardware supports hardware virtualization and we can run KVM on this machine.

For working with KVM guest, this post by Scott Lowe is a good start.

Give KVM a try
Personally, KVM is a very good tool for my home lab.  I believe it is also a good option for commercial deployment of  virtualization or cloud infrastructure.  KVM is the most used hypervisor for OpenStack as of today.  (Note: it seems container is becoming more and more popular and lots of development effort is being put in by individual contributors and vendors to make container "enterprise ready" for the virtualization or cloud infrastructure so let see if container will replace KVM).