OpenStack is now one more step closer to "enterprise" ready

In the month of July, groups around the world the world are celebrating the 5 year birthday of OpenStack. They are giving out  t-shirts to commemorate this event along with birthday cakes and champagne (each group is celebrating a little bit differently). 

OpenStack started in 2010 with is first release named as Austin.  Eric Wright (@discoposse) had a nice post on OpenStack Kilo release and have a detailed description of some interesting statistics about OpenStack as well as some of OpenStack's latest feature.

OpenStack came a long way in these 5 years and is starting to gain more and more attention as a solution to business needs.  Last OpenStack summit in Vancouver there were over 6,000 attendees.  New features in respond to the market needs were being added in a high pace.  While OpenStack is feature rich and is delivering these features in a fast pace with 2 releases per year, one question there is always one question in mind - 

  “Is OpenStack enterprise ready?"

Essential "enterprise" features

I think the top 2 essential enterprise features are:

1. High Availability
2. Security

High Availability
One important aspect for enterprise is to sustain continuous business operation.  To look at a product if it is enterprise ready we must look at what level of HA it is able to deliver.  Different business have different requirement for the level of HA a product can provide. If we look at that OpenStack web site, it stated that OpenStack is able to offer 99.99% of uptime for the infrastructure. But if we read on, it stated that OpenStack does not guarantee 99.99% availability for individual guest instances.  Well it does not mean that OpenStack is NOT enterprise ready.  At least the OpenStack is guaranteed to have 99.99% of uptime.  If this cannot be achieved, then we can forget about the 99.99% of the individual guest instances.

Vendor specific build-in HA features

There are a few ways that we can take advantage of the existing and proven high availability feature provided by vendors such as:

1. Red Hat Enterprise Linux 7 - Pacemaker
2. Ubuntu 14.04 - High Availability Cluster solutions
3. SUSE Linux Enterprise High Availability
4. VMware vSphere infrastructure 

Kubernetes
Another development in OpenStack is project Magnum where OpenStack interacts with Kubernetes to orchestrate the deployment of groups of containers as a Pod and there is the replication controller in Kubernetes that user can defined the desired state of the Pod in such a way that if container dies, the Replication Controller is going to re-create the Pod defined by the user.

image source: https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAjyzgb5tK336Z93gwLMTxC2iSaolGw3SNof_YQZwRa3TrJYuk8OCVBFBuCatAqWhOWedOV7CNto-pB5XCXTe74XIlBFonwUbD-FRveDGpQZIGAwJbsheJPwWk8pb2RUNYC3aKUOV9hYAB/s1600/k8s+replication+controllers+(1).png

Security

Another important aspect for enterprise is security.  OpenStack has a project called Keystone that handles the security of the infrastructure.  One feature for Keystone is that it can use Active Directory as a backend server and this allows OpenStack to use the existing and widely used Active Directory.  This is helping OpenStack to be enterprise ready.

Interaction within the various element of the OpenStack infrastructure is via RESTful API.  Each entity will have to request a security token for each API call so that the target entity can validate if the initiator of the API call has the proper credential and privilege to request the operation.

OpenStack has a dedicated security team to look at the security of the OpenStack infrastructure.  Each commit to the OpenStack code base has a keyword SecurityImpact that the commiter can use to flag the security team to investigate if that particular commit has any security impact to OpenStack.

With all these features in place, more and more companies are willing to deploy OpenStack in the production environment.  This is a sample of companies that are deploying OpenStack in their production environment (just showing some examples and is not a complete list):

 

image source: http://cloudtweaks.com/wp-content/uploads/2014/11/companies-snapshot.png 

Within the last few days there are 2 new development that helped move OpenStack closer to be "enterprise" ready.

Google becoming a sponsor to OpenStack Foundation

On July 16, Google announced that it is becoming a sponsor to the OpenStack Foundation along with “big” companies such as IBM, Red Hat, Ubuntu or VMware support this open source cloud infrastructure.  I had a blog post on my reaction to Google’s announcement.

The significant of Google sponsoring OpenStack Foundation is twofold.  The first one is that Google is in a way endorsing this technology.  The second one is what its knowledge, expertise and experience on deploying container at scale to make OpenStack able to deploy container as well as virtual machine in it infrastructure.

OpenStack Innovation Center

On July 23, there is another OpenStack announcement that is worth looking into and I believe is as important as the Google announcement. This important announcement is that Rackspace and Intel is collaborating to form an OpenStack Innovation Center.  According to the press release the purpose of this OpenStack Innovation Center is to help accelerate the development of enterprise capabilities and thus driving the adoption of OpenStack in the enterprise production environment.

The agreement between Rackspace and Intel includes:

• OpenStack Innovation Center – The center will create the world’s largest OpenStack development team. It will be comprised of Rackspace and Intel engineers and will be located at Rackspace’s corporate headquarters in San Antonio.
• OpenStack Developer Training – Through this effort, Rackspace and Intel will offer new modules of courseware designed to onboard and increase the number of open source developers actively contributing to the success of the community.
• Joint OpenStack Engineering – Rackspace and Intel will resource OpenStack development, working in collaboration with the OpenStack Enterprise Work Group and community, targeting bug elimination and the development of new enterprise features. The companies will recruit new engineers to participate in OpenStack development.
• Largest OpenStack Developer Cloud– Rackspace and Intel will build and make available to the community two 1,000 node clusters to support advanced, large-scale testing of OpenStack and new features developed through the joint engineering work. The companies anticipate having the clusters available for community use within the next six months.

One good news for me as a software developer on this collaboration between Rackspace and Intel is that there will be a two 1,000-node OpenStack hybrid cloud clusters that will be made available to the OpenStack community to look a issues regarding scaling and performance.

What is next?
Wonder what is in the horizon for more enterprise features in OpenStack.  I had a conversation with a person that is in the OpenStack community and he was saying that there will be a lot more exciting announcements in the upcoming OpenStack Tokyo summit and in the OpenStack Austin summit next year. 

Reference:
"Accelerating the Enterprise Features of OpenStack: Rackspace and Intel Form the OpenStack Innovation Center." Rackspace Hosting. N.p., n.d. Web. 25 July 2015.

vSphere 6 – VMware is heading toward VM-aware

VMware vSphere 6 was released in early February 2015.  The must read “Mastering VMware vSphere X” book series for vSphere 6 is already available.  There are tons of blog posts on what is new in vSphere 6 available on the Internet that we can search for.  I had been busy with doing bug fixes for OpenStack right before and after the OpenStack Summit in Vancouver.  My second OpenStack bug fix was submitted upstream and merged last week. It is difficult to have the mind switch between technologies for me.  This is also why I had not published any new blog post lately.  I will try to blog about my experience being an open source code committer in the coming days. 

VMworld 2015 is coming and I predict that there will be lots of new product or feature announcements in the End User Computing area.  Before more new things to learn I have made up my mind to at least catch up with what is new in vSphere 6.

What's new in vSphere 6
The official “what’s new” information page from VMware listed the following:

Compute

•  Increase Scalability
• Expanded Support for new chips sets, devices, drivers and guest OS
• Support for NVIDIA GRID vCPU
• Instant clone

Storage

• VM-aware Virtual Volumes
• Storage Policy-Based Management

Network

• Per-VM Distributed vSwitch bandwidth reservation
• Multicast Snooping (IGMP snooping for IPv4 and MLD Snooping for IPv6)
• Multiple TCP/IP stack for vMotion

Availability

• vMotion Enhancements
• Replication-Assisted vMotion
•  Expanded support for Fault Tolerance (up to 4 vCPUs instead of just one)

Management

• Content Library
• Cross-vCenter Clone and Migration
• Enhanced User Interface

Duncan Epping (@DunccanYB) had a much more detailed summary post on this subject.  For anyone who is involved in VMware related technologies, it is highly recommended to visit his blog "Yellow-Bricks" regularly as there are lots of good contents. According to him the vVol (Virtual Volume) is the “flagship feature” of the vSphere 6 release. I totally agree with Mr. Epping. 

VM-aware is the trend

In fact, if you look at the list of “what’s new”, lots of the items are feature enhancements.  Originally when I write this post the title was “Catching up on what’s new in vSphere 6”.  As I dig into “what’s new”, I see that VMware is making its product VM-aware.  I think it will eventually be heading toward application aware.  I am not an expert in Cisco product (yet).  Cisco is marketing ACI – Application Centric Infrastructure which is also heading toward the same direction.  

“There is no new thing under the sun”, both VMware and Cisco and among some other companies are seeing the need to have the data center infrastructure to be application aware so that we can provide the intelligence to run the infrastructure more effectively.  Another important benefit for the infrastructure to be application aware is – SECURITY. 

At the end of the day, the ultimate goal of having a data center infrastructure is to run business application so that a business entity can earn money.  It is the application that we want it to run efficiently and securely.

To have the infrastructure VM-aware or even application aware, it must be agile so that it can react to the dynamic changes.  vMotion is one example of dynamic changes.

Policy
Software Defined Data Center (SDDC) is the first step of providing support for VM-aware infrastructure.  With software providing an abstraction level to all the elements of the data center, operators/administrators can automation changes as well as to define policies which are the rules of how things should happen according to specified characteristics of a virtual machine.  The technology is still advancing and we can look at the defined policies as being the intelligence of the infrastructure.  The entire data center infrastructure, be it storage or networking reacts to changes according to the defined policies.  One common theme about the characteristics of a policy is that it is “declarative”.  Policy being “declarative” only specifies the end result and not how to attain the result.

Industry Convergence

In the OpenStack world, VMware is investing heavily on a project call “Congress” while Cisco is investing on “Group based policy”, it is interesting to see how the IT industry converge into a common way of providing an infrastructure for business application to run both efficiently and securely.

Reference:
 "VMware Virtualization for Desktop & Server, Application, Public & Hybrid Clouds | United States." VMware Virtualization for Desktop & Server, Application, Public & Hybrid Clouds | United States. N.p., n.d. Web. 03 July 2015.

Navigating through the VMware's forest of product offerings

Back in 2008 when I first started to learn about VMware's server virtualization technology, it is very confusing on the different products and how they are related together.  There are the ESX, ESXi, vSphere and vCenter server and there are older products such as VMware server or the GSX server.  With Google we can find lots of articles on these products but the problem is that most of these articles do not have a date and since the VMware virtualization technology is advancing in such a high pace, we do not know if the information was current or relevant to which version of vSphere.

Now in 2015, can you tell me all the current VMware products and how they are related?

To look at VMware's product offering, it is best to look at VMware's vision on data center - Software Defined Data Center architecture.

Data Center comprises of these functions:

1. Compute
2. Storage
3. Network
4. Management

VMware’s idea is to provide an abstraction layer for compute, storage and network hardware so that the entire data center is software driven.   

The advantage of a Software Defined Data Center architecture is that the data center can be more agile as the hardware resources are provisioned or withdraw on demand to the compute, storage and network resources.  This agility factor has huge implication to IT operations in a data center:

• More cost effective
• Ability to service user demand much faster
• Provide a stable platform for DevOps
• Eliminate or minimize human error
• Make Disaster Recovery easier thus able to improve RTO and RPO.

The benefits of a Software Defined Data Center architecture is not limited to the above mentioned points and it is not the main point for this article.

VMware vCloud Suite is VMware's solution to provide a "Software Defined Data Center" for customers. As shown in the diagram below vCloud Suite comprises of:

• Compute
• Storage and Availability
• Management and Automation
• Network and Security (add on feature)

Each of these different products under the above mentioned categories can be deployed separately and as a whole they formed the vCloud Suite.

image source: http://www.vmware.com/files/images/thumbnails/vmw-scrnsht-vcloud-suite-mgmt-lg.jpg

Compute
At the heart of the vCloud Suite is the vSphere Suite that provide the virtualization infrastructure for different business establishments from SMB to enterprise to cloud operators. I don't think I need to describe much about the vSphere Suite as this has been around for a number of years and are widely deployed.

Management and Automation

The VMware vRealize™ Suite provides the management and automation capability for the Software Defined Data Center architecture.

Basically, the vRealize Suite comprise of Automation, Operations and Business and Log Insight for monitoring.

image source: http://cloudmaniac.net/wp-content/uploads/2014/10/vmware-vrealize-suite.jpg

Note:

• vRealize Operations is formerly known as vCenter Operations Management Suite or vCops
• vRealize Automation is formerly known as vCloud Automation Center or vCAC
• vRealize Business is formerly known as IT Business Management Suite
• vRealize Log Insight is formerly know as vCenter Log Insight

For a more comprehensive and detailed description of the vRealize Suite, this VMware site is a good place to visit.

VMware vRealize Suite consists of the following products:

• VMware vRealize™ Automation™ Advanced or Enterprise
• VMware vRealize™ Operations™ Advanced or Enterprise
• VMware vRealize™ Log Insight™
• VMware vRealize™ Business™ Standard
• VMware vRealize™ Business™ Advanced or Enterprise

Storage and Availability
For all IT operations, disaster recovery is an important element.  Each business should defined the Recovery Time objective (RTO) and Recovery Point Objective (RPO) for acceptable level of disruption in case of a disaster event affecting the continuous operation of the IT infrastructure. Most importantly DR plans needs to be tested.  VMware vCenter Site Recovery Manager provide customer the ability to automate and orchestrate non-disruptive testing of recovery plans. With automation and orchestration, the RTO and RPO can be improved.

image source: http://ddf912383141a8d7bbe4-e053e711fc85de3290f121ef0f0e3a1f.r87.cf1.rackcdn.com/2.17%20vcenter.png

For more comprehensive and detailed description of VMware vCenter Site Recovery Manager visit here.

I am not sure why VMware group storage and availability into the same category because availability also include the compute and networking pieces that support the application that we are ultimately concern with.

Regarding storage, VMware had been working on abstracting the storage hardware and to provide an easy to configured Software defined Storage products.  In this area, VMware is taking 2 approaches:

1. Virtual Data Plane - Virtual SAN (VSAN) and vSphere Virtual Volume (vVol)
2. Policy Driven Control Plane - Storage Policy Based Management

There are many other blog post talking about these products. This blog post is meant to provide an overview and how these products related to each other.

Network and Security
This is an add-on to to the vCloud Suite.  In this category is the NSX - VMware's way of providing an abstraction layer to the physical network as well as some L4 - L7 network services.

NSX is also a security product as I have talked about in my previous post.

Brad Hedlund (@bradhedlund, Engineering Architect at VMware's Network and Security Business Unit) has an good article on what network virtualization is

 

image source: http://commondatastorage.googleapis.com/bradhedlund/blog/what-is-network-virtualization/What_is_Network_Virtualization.PNG

NSX virtualized the network as well as network functions such as load balancing, firewall and VPN. Any Cloud management platform including OpenStack can interact with NSX via RESTful API to provision network services for the cloud platform.  

NSX has two flavors as I have outlined in this post.  One flavor is tailor toward vSphere product and the other one is for non VMware hypervisors such as KVM.  

One point worth mentioning is that NSX can optimize the VXLAN multicast traffic because the NSX Controller is able to distribute the MAC address and VXLAN ID mapping to the various VTEPs instead of using bi-directional PIM as defined in RFC 7348 (I have a summary of this RFC here).  Multicast traffics takes up network bandwidths and network administrators always try to find ways to eliminate multicast traffic.

Other VMware products

For a complete list of VMware products we can find them here. In this post I have not talked about the End-user computing product of VMware - Horizon nor the VMware hybrid cloud product - VMware Air which is the VMware's offering of public cloud similar to Amazon's AWS, Google's Cloud platform or Microsoft's Azure.

I hope this will clarify VMware's product offerings and how they relate to each other.