A project on Data Plane Development Kit

Project Title

A VPP plugin utilizing Intel® DPDK and QuickAssist Technology to perform hardware assisted compression operations.

Project Description

The Intel® QuickAssist Technology is a powerful hardware based solution to perform crypto and compression operations. QuickAssist offloads the operations from CPU to the 89XX communications chip. A VPP plugin that utilizes the QuickAssist feature for data compression can be used as a graph node and can be called by the packet processing graph.

Acceleration Enhancements for DPDK currently already has the ability to perform cryptographic operations either by software or hardware depending on the capabilities of the processor that the code is running on. This project will be adding compression to the Acceleration Enhancements for DPDK and at this time limit to hardware based only while later only add in software based compression similar to the crypto counter part.

This VPP plugin will use this compression feature that will be added to the Acceleration Enhancements for DPDK to perform the compression operation.

Below are 2 diagrams that I get from the Internet and included in the proposed project in Intel Developer Mesh for clarity:

image source: https://wiki.fd.io/images/c/c9/VPP_custom_application_packet_processing_graph.280.jpg

image source: http://images.slideplayer.com/39/10926601/slides/slide_5.jpg

Project Use Case

IP payload compression (RFC 3173) that help to save bandwidth can make good use of this hardware assist technology to speed up the compression operations.

Getting Started

To start this project I first looked at what Intel’s QuickAssist Technology is. Of course Google is the first place I went. I found this article from Admin Magazine very helpful in getting me started. It helped me to have a general understand of the features and a high level understand on how QuickAssist Technology works. One thing I like about this article and found it useful is the block diagrams and also it explains how this QuickAssist Technology can boost performance of different use cases and of course NFV is what I care about the most.

image source: http://www.admin-magazine.com/var/ezflow_site/storage/images/archive/2016/33/an-introduction-to-intel-quickassist-technology/figure-3/125991-1-eng-US/Figure-3_reference.png

Next this web site provides tons of useful resources and most of all it contains the source file for the Linux Driver for the hardware that supports this QuickAssist Technology as well as the programmer’s Guide along with the Cryptographic and Compression API Reference Manual.

This article is also a good resource in understand the uses cases of QuickAssist Technology and Network functions.

With this in place, I download the DPDK source code and the DPDK Programmer's Guide as well as the Getting started Guide.

Initially for this proposed project my idea was that currently DPDK already had the QAT related PMD (Pull Mode Driver) and a sample program and thus it should be simple to make "some" modification to add in compression support.  Both the Crypto PMD driver and dpdk-qat sample application has useful documentations.

With all the source code and documentations, I am all ready to go.

Hitting a Road Block

Out of my surprise, currently everything in the DPDK code base is geared specifically on crypto and no mention of compression. I found myself hitting a road block. I am not sure if I should modify the existing Crypto QAT driver or come up with a Compression QAT driver. First of all I see that this is a design decision and also being new to DPDK, I am not quite ready to write a new Pull Mode Driver from scratch.  I need more time and information.

Reaching out to the DPDK Community for help

I then reach out to the DPDK developer mailing list but unfortunately, this is over the Christmas holiday and only after 3 days that I got one reply saying compression is not currently supported as well as point me to a DPDK QAT documentation.

Another day passed, still no further reply on my query and I sent out another Email to the mailing list. This time I also mention not getting any help from the DPDK mailing list. In that afternoon, I get a reply from an Intel program manager. He told me his team worked on crypto portion of QAT and kept me engaged over Email.After explaining to him where I am coming from he said most of his team member are on vacation and when they come back in January 2017, they will see if they have the resources and priority to get this going and is happy to include me in making this to work.

Project Scope needs adjustment

It seems that the scope of this project is bigger than I have expected and takes longer time and more collaboration of the DPDK community to complete.

Current Status of this Project

While waiting for the DPDK design decision and collaboration, I am currently  looking at building a VPP plugin.

I am only able to complete this much by December 30, 2016. There is no code to show and nothing to demo yet.

I, however, even not able to finish this project before 2016, I am going to complete this proposed project in 2017. Will update the latest status as I move along.

Are you interested in working on this project?

Join me in this project if you are interested and make it to completion. Contact me at the Intel Dev Mesh website or ping me on Twitter - @vCloudernBeer.