vSphere 6 – VMware is heading toward VM-aware

VMware vSphere 6 was released in early February 2015.  The must read “Mastering VMware vSphere X” book series for vSphere 6 is already available.  There are tons of blog posts on what is new in vSphere 6 available on the Internet that we can search for.  I had been busy with doing bug fixes for OpenStack right before and after the OpenStack Summit in Vancouver.  My second OpenStack bug fix was submitted upstream and merged last week. It is difficult to have the mind switch between technologies for me.  This is also why I had not published any new blog post lately.  I will try to blog about my experience being an open source code committer in the coming days. 

VMworld 2015 is coming and I predict that there will be lots of new product or feature announcements in the End User Computing area.  Before more new things to learn I have made up my mind to at least catch up with what is new in vSphere 6.

What's new in vSphere 6
The official “what’s new” information page from VMware listed the following:

Compute

•  Increase Scalability
• Expanded Support for new chips sets, devices, drivers and guest OS
• Support for NVIDIA GRID vCPU
• Instant clone

Storage

• VM-aware Virtual Volumes
• Storage Policy-Based Management

Network

• Per-VM Distributed vSwitch bandwidth reservation
• Multicast Snooping (IGMP snooping for IPv4 and MLD Snooping for IPv6)
• Multiple TCP/IP stack for vMotion

Availability

• vMotion Enhancements
• Replication-Assisted vMotion
•  Expanded support for Fault Tolerance (up to 4 vCPUs instead of just one)

Management

• Content Library
• Cross-vCenter Clone and Migration
• Enhanced User Interface

Duncan Epping (@DunccanYB) had a much more detailed summary post on this subject.  For anyone who is involved in VMware related technologies, it is highly recommended to visit his blog "Yellow-Bricks" regularly as there are lots of good contents. According to him the vVol (Virtual Volume) is the “flagship feature” of the vSphere 6 release. I totally agree with Mr. Epping. 

VM-aware is the trend

In fact, if you look at the list of “what’s new”, lots of the items are feature enhancements.  Originally when I write this post the title was “Catching up on what’s new in vSphere 6”.  As I dig into “what’s new”, I see that VMware is making its product VM-aware.  I think it will eventually be heading toward application aware.  I am not an expert in Cisco product (yet).  Cisco is marketing ACI – Application Centric Infrastructure which is also heading toward the same direction.  

“There is no new thing under the sun”, both VMware and Cisco and among some other companies are seeing the need to have the data center infrastructure to be application aware so that we can provide the intelligence to run the infrastructure more effectively.  Another important benefit for the infrastructure to be application aware is – SECURITY. 

At the end of the day, the ultimate goal of having a data center infrastructure is to run business application so that a business entity can earn money.  It is the application that we want it to run efficiently and securely.

To have the infrastructure VM-aware or even application aware, it must be agile so that it can react to the dynamic changes.  vMotion is one example of dynamic changes.

Policy
Software Defined Data Center (SDDC) is the first step of providing support for VM-aware infrastructure.  With software providing an abstraction level to all the elements of the data center, operators/administrators can automation changes as well as to define policies which are the rules of how things should happen according to specified characteristics of a virtual machine.  The technology is still advancing and we can look at the defined policies as being the intelligence of the infrastructure.  The entire data center infrastructure, be it storage or networking reacts to changes according to the defined policies.  One common theme about the characteristics of a policy is that it is “declarative”.  Policy being “declarative” only specifies the end result and not how to attain the result.

Industry Convergence

In the OpenStack world, VMware is investing heavily on a project call “Congress” while Cisco is investing on “Group based policy”, it is interesting to see how the IT industry converge into a common way of providing an infrastructure for business application to run both efficiently and securely.

Reference:
 "VMware Virtualization for Desktop & Server, Application, Public & Hybrid Clouds | United States." VMware Virtualization for Desktop & Server, Application, Public & Hybrid Clouds | United States. N.p., n.d. Web. 03 July 2015.

VMware NSX and HP VAN SDN Controller - a value added solution

I am a software developer and for a long time when I implement a feature SNMP support for the feature is always done last and sometime delayed to a future release.  Back then, functionality of the feature was the most important thing in my mind.  As I venture into server virtualization, I start to see the perspective of a system administrator or a network administrator.  Monitoring and reporting is very important and sometimes more important than a feature set delivered by the vendor.  Of course maintaining the five 9s of uptime is always the highest priority.

These days I am looking into NSX.  Derek Seaman has a blog post on “VMworld 2014: Future Direction of NSX” where he summarize session NET1674.  Chris Wahl has a post on NSX 6.1 (4.2 for the NSX Multi-hypervisor).

I came across an article with the title “The industry’s east-west federated solution”.

After reading the title, 2 questions come to my mind.  What is:

• East-West traffic?
• The solution for what problem?

East-West Traffic

In simple term, east-west traffic refers to the traffic between servers in a data center.  There is another term – north south traffic and this refers to the traffic between clients and servers in a data center.  Traffic from client to the server will be northern bound traffic while traffic from server to the client is called southern traffic.

Image source: http://regmedia.co.uk/2011/08/15/xsigo_east_west.jpg

A Solution to a problem

First of all what is this solution?  What problem is this trying to solve?  The problem is that virtual network has not visibility to the physical network.  For a user or virtualization/network administrator this is not a big problem.  This will just be an inconvenience because there are tools to monitor, view and debug the virtual and physical network individual.  At the end of the day, user still has the tools to perform their job.

From the perspective of automation or orchestration, this is a big problem. 

A Federated Solution

Why federated?  The solution calls for a federation of 2 products to solve the problem.  The 2 products are VMware’s NSX and HP’s VAN Controller which one of HP’s SDN solution.

VMware NSX

Tons of information can be found about VMware’s NSX.  Not too many people have the luxury of having the opportunity to play around with this product.  In VMworld 2014, VMware announced a new certification track for Network Virtualization which has high NSX concentration.  Information on this certification track can be found here.

Image source: http://blogs.vmware.com/education/files/2014/08/VMW_14Q3_Cert_Roadmap_Network_Virtualization_R3.jpg

As of this writing, I see on twitter that a few people had sit for the VCP-NV certification and a few already achieved VCDX-NV status and they we being introduced on VMworld 2^nd day Keynote session.  The VCIX-NV is not available yet and this should be equal as getting a VCAP level certification and one of the requirements to sit for this certification is a person is a CCNP or CCIE holder.  This requirement shows that VMware is trying to get Cisco certification holders to get into VMware’s SDN solution.

When we look at VMware NSX, we can approach this from 2 angels.  Its capabilities and its components

VMware’s NSX has the following Capabilities:

• Logical Switches
• Logical Routers
• Logical Firewall
• Logical VPN
• Logical Load Balancer

VMware’s NSX has the following components

• NSX Manager
• NSX vSwitch
• NSX Controller
• NSX Edge

All the NSX components can be configured using vSphere Client, VMware command line interface (CLI) and REST API. 

The REST API is essential for 3^rd party software entities to interface with NSX.

HP’s SDN Technology

This page contains a very comprehensive description of HP’s SDN technology.  One of HP’s SDN offering is HP Virtual Application Network SDN Controller.  It is described as a control point in an OpenFlow-enabled network, simplifying management, provisioning, and orchestration. This enables delivery of a new generation of application-based network services and provides open application program interfaces (APIs) that allow third-party developers to deliver innovative solutions to dynamically link business requirements to network infrastructure via either custom Java programs or general-purpose RESTful control interfaces. HP VAN SDN Controller Software is designed to operate in campus, data center, or service provider environments

This HP VAN SDN controller is recommended to run on an Ubuntu 64-bit server with 12.04 LTS.  It requires a back end database.  The recommended database platform is PostgreSQL 9.1.  OpenJDK 7 JVM is also required.

Integrating VMware and HP’s solution

The 2 companies has entered into a joint development of a solution that will combine the strength of the 2 SDN solutions that according to a white paper to provide customers unified automation and visibility of the physical and virtual data center networks, enabling  business agility and improving business continuity.

Image source: https://pbs.twimg.com/media/BwJ2YaFIEAItUY9.png:large

The integration of the 2 controllers is using OSVDB (Open vSwitch Database Management Protocol).  The HP controller is acting as an OSVDB server while the VMware NSX controller is acting as an OSVDB client.  Both controller are able to communicate with each other via the OSVDB federated API.

HP blog is saying the integration is at the control plan.  When I look into OSVDB, I believe this should be the combination of the management and control plan where OSVDB is for management similar to NETCONF and control plan is where the OpenFlow protocol resides in which is being use to program traffic flows.

This solution will be available in the 4^th quarter of 2014.

VXLAN in the contemporary data center

What is a Contemporary data center?

A contemporary data center is a virtualized data center.  At first it was only the server that was virtualized.  Virtualizing the servers alone changed the data center from static environment to a dynamic environment where servers running as virtual machines can be provisioned and deleted as well as moved from one physical machine to another.  The contemporary data center has changed into a dynamic/elastic environment.  

Later on, storage virtualization has made the data center more dynamic/elastic where beside the virtual machines, the data can be moved around. 

Virtual machines can move from one physical server to another server is very useful.  However, the limitation was that these physical servers have to be connected in a flat network (layer 2).

With multiple virtual servers running on a physical server allows for multi tenancy.  VLAN is a good way for traffic isolation among the various tenants.  The number of VLANs in a network is limited by the 12-bit field which is 4096 in which VLAN 0 is not a valid VLAN thus only 4095 VLANs can exist in a given layer-2 network.

To cope with the increased demand on the network from the virtualized data center the industry has come up with 3 different ways to alleviate the problems.  The most discussed technologies are:

• Network Virtualization
• Network Function Virtualization and
• Software Defined Networking

I will describe and compare these 3 technologies in another post.  This post will focus on VXLAN which is one version of Network Virtualization.

What is Network Virtualization?

Virtualization is the abstraction or decoupling of something from the physical entity.  In this case, for network virtualization it is the ability to abstract networking from the physical network. 

How does network virtualization abstract from the physical network?  One way is to use the technique of network overlay where tunnels between end points are created on existing physical networks.  The most common tunneling protocols are:

• VXLAN (Virtual Extensible LAN)
• Network Virtualization using Generic Encapsulation (NVGRE)
• Stateless Transport Tunneling (STT)
• Network Virtualization Overlay3 (NVO3)

Benefits of Network Overlay

The very first problem that network overlay can help solve is to extend the layer 2 domain across layer 3 subnets.  This resulted in physical servers are not confined to a single flat layer 2 network for virtual machines to move around.  With traffic tunneled between end points, it helps in traffic isolation among tenants.

Each overlay networks has its own network id and thus extend the 4095 VLAN limitation.  Furthermore, multi-tenants in the same data center can have the same private IP address.

What is VXLAN?

VXLAN (Virtual Extensible LAN) is a network tunneling technology by encapsulating UDP packet on top of a native Ethernet frame and transport over an IP network. 

It was jointly developed by VMware, Arista Networks and Cisco.  The latest specification which is moved to RFC status also has contribution from Storvisor, Broadcom, Citrix and Red Hat.  The title of this IETF draft is “VXLAN: A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks”.  This technology was first announced at VMworld 2011. Since then there are tons of articles about this topic on different technical magazine and blog.

VXLAN Terminology

The best way to understand a technology, in my opinion is to start from the terminology being used.  It provides a framework of what the important elements are for a given technology. At the very least we can type in these key words in our favorite search engine and start researching.

The following are in my opinion the essential basic terminology used in the VXLAN world:

• Encapsulation
• VTEP
• VNI
• IP Multicast 

Encapsulation

The term encapsulation is used in Object Oriented Programming as well as in data communication.  The idea is the same in both cases. The concept of encapsulation is to put one object into another object and send to a destination. 

In the case of VXLAN, it is to put a layer-2 frame as the payload of an UDP packet and uses IP to reach the destination.  When reaching the destination, the packet is being de-capsulated.

I have a picture taken from the Cisco website that not only details the individual field of a VXLAN packet but also to explain the concept of encapsulation with color.  The yellow portion is the “original L2 frame” and is being put as the payload of an UDP packet as highlighted in blue.

Image source: http://www.cisco.com/c/dam/en/us/products/collateral/switches/nexus-9000-series-switches/white-paper-c11-729383.doc/_jcr_content/renditions/white-paper-c11-729383-02.jpg

VTEP

As we described in the above paragraph, packets are being encapsulated and de-capsulated from the source to the destination.  VTEP (VXLAN Tunnel End Point) is the entity that performs the encapsulation and de-capsulation. 

VTEP plays a vital role in the VXLAN operation.  It is these end points that the tunnel is created so that the “original L2 frame” can be transported back and forth thus achieving the goal of layer-2 communication over a layer-2 (IP) infrastructure when entities are in different IP subnets.  One constraint that we have mentioned in this post was that vMotions of virtual machine is limited to physical machines that are in the same layer-2 network.

VTEP can be implemented in virtual switches in the hypervisors or it can be on physical networking device such as switch and routers.

VNI

VXLAN is about layer-2 segments as inferred by its name – extensible.  Traditional VLAN segment is limited by the 12-bit VLAN ID to 4096 per network.  With VXLAN it is being expanded to 16 million logical segments.  This is done by the use of a 24-bit VNI or VNID (VXLAN Network ID) to uniquely identify logical segment within a VXLAN network.

Each device in the VXLAN network is uniquely identified by the combination of VNI and the MAC address.

IP Multicast

VXLAN operates on a layer-3/IP network.  Tunnels are created between VTEPs. IP multicast was specified in the VXLAN specification to be used to simulate a layer-2 broadcast to find the location of the destination device.  IP multicast can be IGMP or PIM.   I will have to find out if one method is being used more than the other.

Cisco has a proprietary implementation of using unicast to perform this function.  It is being called the unicast-mode.

Putting the terminology together to see how VXLAN works

Knowing the terminologies is like knowing the alphabets.  Now we are to make a sentence from the alphabets.

Image source: http://blogs.vmware.com/vsphere/files/2013/05/Learning-1.jpg

The above diagram that I have seen at the VMware blog explains the VXLAN operation very well:

• VTEP is implemented in VMware’s vSphere Distributed Switch.
• VTEP has an IP address and in this case they are on the same subnet.
• There is a Layer-3 infrastructure network.
• The 2 VTEPs are member of a multicast group and in this case IGMP (Internet Group Management Protocol) is used.
• The VNID is 5001.

When VM on the left wanted to communicate with VM on the right:

• VM sends out a destination unknown, broadcast or multicast packet
• VTEP on the left (IP = 10.20.10.10) encapsulate this layer-2 frame into an UDP packet and send it out to the multicast group.
• Other VTEPs in the multicast group (in this case there is only one) received the packet will de-capsulate and flood the packet on their local layer-2 domain.
• In this process the VNI and the MAC address of the VM on the left is learned by the VTEPs.
• VM on the right received the frame from VM on the left and reply.
• The reply frame will be send from VTEP on the right to the VTEP on the left as a unicast frame since the MAC address and VNI are learned.
• At this time the MAC address and the VNI of both VMs are learned by the VTEP and from this point onward, traffic between both VMs will be IP unicast between the 2 VTEPs

This is a brief overview of VXLAN in the contemporary data center.