VMware – from VM-aware to Application-aware

After catching up with what’s new in vSphere 6 this past weekend, I started to catch up with VMware’s latest announcements.  One of the latest announcements is “Project Bonneville”.  This is project is in the technology preview state and I am hoping that there will be some exciting new announcements on this in the upcoming VMworld 2015.

I think you are all familiar with this logo.  Docker Inc, is the company behind the open source Docker platform.  The Docker platform is an orchestration or packaging tool that allows application and their dependencies to run on container technology.  Container is not a new technology ad we can trace back the origin to FreeBSD Jails back to year 2000. Docker makes it easy for developer and sys admin to deploy application with container technology.  This is just perfect for DevOps as well as Microservices architecture. 

Many of us buy things at www.amazon.com and it is very convenient especially if we also subscribe to the Amazon Prime services.  I ordered my “Mastering VMware vSphere 6” book from Amazon and it was on my desk in one and a half day.  In the pass the web sites for online shopping follow the LAMP stack model with Linux, Apache, MySQL and PHP.  Nowadays the with Docker making the container much more user friendly, there is a new way of deploying an online shopping website with an Microservices model.  The entire website is supported by a bunch of Microservices that has it specific function/purpose.  Each Microservice runs as a container and interface with each other. 

Often time the general view is that when Docker/container technology is becoming more and more mature virtual machine will be phased out.  VMware, however, see this very differently.  Last year at VMworld 2014, it announced project Fargo (now shipped as Instant clone in vSphere 6 for rapidly clone and deploy of virtual machine) where it sets the direction of container at VMware:

VMware + Docker = Best of both worlds.

I had a blog post that talks about this and if you are not familiar with Docker or container technology take a look here as as start.

Benefits of running a VMware infrastructure
I believe you can mention more benefits of running a VMware infrastructure than what I can list out in this post.  It is a matured product and with constant feature enhancements.  It is full of enterprise ready features.

The foremost benefits of a VMware infrastructure I think will be the resource management capability such as vMotion, Distributed Resource Scheduling (DRS)  and built-in fault tolerance.  Monitoring and reporting capabilities will be another benefits.  Last but not least is the rich networking feature that comes with vSphere as well as NSX that support both networking and security for the VMware infrastructure.

Benefits of running Docker containers
These days, more and more companies are running Docker containers in the production environment.  As mentioned earlier, Docker containers are good for DevOps and Microservices architectures due to is lightweight footprint and minimal overhead which make deployment extremely fast.  Another benefit of a Docker container is that it bundles the application and all its dependencies into a single image and can be run on any machine as well as able to share the image with others with the use of a common accessible repository - Docker Hub.

Project Bonneville

This project in a nutshell is to run Docker container as a virtual machine on an ESXi host.

A picture is worth a thousand words and let's take at look at a presentation from VMware on how Docker fits into ESXi host:

image source: http://venturebeat.com/wp-content/uploads/2015/06/VMware-Project-Bonneville.png

It makes perfect sense to run a Docker container on an ESXi host because it can take advantage of the secure and build-in high availability VMware SDDC infrastructure plus the lightweight, fast startup of a container.  The strong point of VMware is where Docker container is the weakest at this time (at least for now security and networking is where Docker containers need to strengthen the most for it to be enterprise ready)

In OpenStack, the trend is also to run containers just like any other hypervisors in Nova compute.  There is also a OpenStack project - Magnum that integrate container orchestration tools such as Docker or Kubernetes for OpenStack users to deploy containers in the cloud.

Visit here and here for a more detail on Project Bonneville,

Moving toward Application-aware
With Project Bonneville, VMware is making a Docker container running as a virtual machine.  In my last post I have mentioned that VMware is moving toward VM-aware infrastructure and now application can be run as virtual machine and thus VMware is able to be define application-aware policies to facilitate efficient and secure infrastructure.

At the end of the day it is the Application that matters
I have mentioned this again and again that Application is the focus of an infrastructure because the main goal of every infrastructure is to deploy application for various business needs.  Email is a good example of application.  Different IT vendors will converge into an "Application Centric" infrastructure however their marketing department will come up with creative name to describe it.

Docker – A Technology that VMware also embraces

Often time there is the notion that with Docker, the Linux Container technology is going to replace server virtualization.  The reasoning for this thought is that with Linux Container virtualize application on the operating system level in which the hypervisor is no longer needed. 

Another camp of thought is that container does not have the robustness and enterprise ready feature such as resource allocation management, high availability or even manageability that can be offered by VMware.

In VMworld 2014, VMware CEO Pat Gelsinger on his keynote session announced the collaboration of VMware, Google and Docker in the Software Defined Data Center (SDDC) sighting that running Docker on a virtual machine is the best of both world giving user a lot more flexibility and benefits.

What is Docker

Docker, Inc is the company behind the open source Docker platform.

Docker is an orchestration or packaging tool that allows applications and their dependencies to run on container technology. 

According to this article Docker consist of:

• Docker Engine – a portable, lightweight run time and packaging tool.
• Docker Hub – a cloud service for sharing application and automating workflow.

It is all about:  APPLICATION.

Before we dig deeper into Docker we have to look into the container technology.

Container

Container is not a new technology.  We can trace the origin to FreeBSD Jails back to year 2000 where programs are run in a sandbox.  Solaris (now part of Oracle) has this implemented as zones.

We can look at container as operating system level virtualization in which application in a container is isolated from each other but are running on the same operating system which is on a single host. 

Google has its own version of container – Imctfy and is being used heavily to support Google Search Engine, Gmail and other Google applications.

Native Linux has container build upon cgroup and namespaces but it is not so easy to deploy application with LXC and thus this technology is not popular in the enterprise space.  Docker is making this much easier both for developer and sys Admin to deploy application with the container technology. 

Once an application is “Dockerized”, it can be run on any platform as long as the OS is the same as the container is created.  Now we can deploy container on-premises (private cloud) or move it to the public cloud such as Amazon Web Services or Google Cloud Computing.

Recently Microsoft has announced support for Docker in their public cloud – Azure.

Popular Configuration Management Tools such as Puppet and Chef can work with Docker and this made the deployment process even easier and Docker a perfect fit for DevOps.

Docket support can also be found in OpenStack Nova.

Docker Components and Technologies

Docker operates on a client and server model.  The Docker client and server/daemon can be on the same host or different host. The Docker client communicates with the Docker server/daemon using REST API.

This diagram captures the core components of Docker:

Image source: http://blog.arungupta.me/wp-content/uploads/2014/07/docker-architecture-techtip39.png

Docker Client

• Accept commands from the user and communicate with the server/daemon

Docker Server/Daemon

• Building the Docker container from the images that are stored in the Docker Registry

Docker container

• Base unit where the application runs on
• Similar to a Virtual Machine

Docker image

• Building block of container

Docker Registry

• Location where the Docker images are stored
• Public registry – access by everyone
• Private registry – access by specific team or organization

Red Hat has a good description of Docker fundamental components:

·     Container – an application sandbox. Each container is based on an image that holds necessary configuration data. When you launch a container from an image, a writable layer is added on top of this image. Every time you commit a container (using the docker commit command), a new image layer is added to store your changes.

·      Image – a static snapshot of the containers' configuration. Image is a read-only layer that is never modified, all changes are made in top-most writable layer, and can be saved only by creating a new image. Each image depends on one or more parent images.

·      Registry – a repository of images. Registries are public or private repositories that contain images available for download. Some registries allow users to upload images to make them available to others.

·      Dockerfile – a configuration file with build instructions for Docker images. Dockerfiles provide a way to automate, reuse, and share build procedures.

Along with the components let’s take a look at the technologies that make Docker works:

Namespaces

Linux namespaces provides isolations for each container.  Applications or process inside a container do not have access outside of the namespaces that the container is in.  There are different namespaces and examples are pid, net, ipc, mnt or uts.

Control groups/cgroups

While namespaces provides access isolation, the control groups limits the hardware resources that the container can access.  One example of control groups is to limit the memory available for the container for say 256 MBs.

UnionFS

This is how containers are made to be light weighted.  Linux kernel first mounts the root system read-only and then change to read-write.  With the union mount, instead of changing from read-only to read-write, a read-write file system is layered on top of the read-only based Filesystem. Union mean to layer read-write with read-only layers.

Image source: https://docs.docker.com/terms/images/docker-filesystems-multilayer.png

Containers

Linux container (LXC) is an essential technology that Dockers uses. 

VMware Project Fargo

So how does VMware embrace Docker?  If you want more information about how VMware uses Docker this blog post is a good start.

In VMworld 2014, VMware announced Project Fargo (currently in beta as of this posting).

Project Fargo according to the blog post from the blog post mentioned is “a technology to provide a fast, scalable differential clone of a running VM” and it is particular useful in a VDI environment.  In fact, it is to make Docker containers to run faster on a VM than they are running on a native Linux machine.   

This is how VMware is saying: VMware + Docker = Best of both worlds.

More information about Project Fargo can be found here, here and here.