DPDK Support | PcapPlusPlus (original) (raw)

DPDK

The Data Plane Development Kit (DPDK) is a set of data plane libraries and network interface controller drivers for fast packet processing, currently managed as an open-source project under the Linux Foundation. The DPDK provides a programming framework for x86, ARM, and PowerPC processors and enables faster development of high speed data packet networking applications (taken from Wikipedia).

DPDK provides packet processing in line rate using kernel bypass for a large range of network interface cards. Notice that not every NIC supports DPDK as the NIC needs to support the kernel bypass feature. You can read more about DPDK in DPDK's web-site and get a list of supported NICs here.

As DPDK API is written in C, PcapPlusPlus wraps its main functionality in easy-to-use C++ wrappers which have minimum impact on performance and packet processing rate. In addition it brings DPDK to the PcapPlusPlus framework and APIs so you can use DPDK together with other PcapPlusPlus features such as packet analysis, etc.

What does PcapPlusPlus offer for DPDK?​

PcapPlusPlus tries to cover the main functionality of DPDK and its most important and popular features. Here is what PcapPlusPlus offers for DPDK:

Set up PcapPlusPlus with DPDK​

Supported DPDK versions​

The following DPDK versions are currently supported:

Older and newer versions may not work. The non-LTS versions may also work, but are not regularly tested.

In addition, not all poll-mode drivers (PMDs) were tested, but the majority of them should work. Please report an issue if the PMD you're using isn't working.

The following operating systems and configurations were tested:

Download and install DPDK - prior to DPDK 20.11​

Building and installing DPDK is pretty straight-forward and in a nutshell goes like this:

$ cd /dpdk/source/location
$ make config T=[platform_type_string] && make

Download and install DPDK - DPDK 20.11 forward​

DPDK changed their whole build system in version 20.11 and is now using meson and ninja. The full build instructions can be found here, but here are the important steps in short:

$ meson build
$ cd build
$ ninja
$ ninja install

Build PcapPlusPlus with DPDK​

Once the DPDK build is completed successfully, build PcapPlusPlus with DPDK using the -DPCAPPP_USE_DPDK option. Please refer to the build instructions for more details.

DPDK initialization with PcapPlusPlus​

DPDK has two steps of initialization: one that sets up Linux to run DPDK applications and the other at the application level which initializes DPDK data and memory structures. PcapPlusPlus wraps both of them with easy-to-use interfaces. Please see the following two sections to get more information.

Initialization before application is run​

Several Linux configuration steps are needed to run DPDK applications:

PcapPlusPlus offers a python script that automatically configures all of the above. The script is located in CMake build directory (e.g /PcapPlusPlus/build) and is named setup_dpdk.py. It is based on the dpdk-devbind tool that ships with DPDK and extends it with more functionality. The script supports Python 3+.

This script has 3 modes of operation: setup to configure the steps mentioned above, status to view the status of all known network interfaces, and restore to go back to the original Linux configuration.

tip

Note: this script uses another file named setup_dpdk_settings.dat to keep information needed for restoring the Linux configuration. This file is also located in PcapPlusPlus root directory. Please do not remove this file

Setup - takes the following parameters:

Parameter Description
-g, --huge-pages AMOUNT The amount of huge pages to allocate. By default each huge-page size is 2048KB
-i, --interface NIC_NAME [NIC_NAME ...] A space-separated list of all NICs that should move from Linux to DPDK control. Only these NICs can be used by DPDK, the others will stay under Linux control
-m, --dpdk-module {igb_uio,uio_pci_generic,vfio-pci} The DPDK module to load. If not specified the default is igb_uio. NOTE: in DPDK 20.11 igb_uio was moved outside of the DPDK repo into a separate repo which means it doesn't come out of the box and needs to be built separately. However the recommendation is to use either vfio-pci or uio_pci_generic which come as part of most Linux distros. You can read more about it here
-k, --load-kni Install the KNI kernel module (not loaded by default)
-p, --kni-params KNI_PARAMS Optional parameters for installing the KNI kernel module
-r, --rte-sdk RTE_SDK DPDK home directory. If this argument is used, the script will use it as the DPDK directory, otherwise it will look for the RTE_SDK environment variable. If both don't exist it will look for RTE_SDK in setup_dpdk_settings.dat. If none of them exist an error will be thrown
-v, --verbose Print more verbose output

If everything goes well the system should be ready to run a DPDK applications and the output should look something like this:

pcapplusplus@ubunu:~/PcapPlusPlus$ sudo python setup_dpdk.py setup -g 512 -i enp0s3
[INFO] set up hugepages to 512
[INFO] loaded kernel module 'uio'
[INFO] loaded DPDK kernel module 'igb_uio'
[INFO] set interface 'enp0s3' down
[INFO] bound interface 'enp0s3' ['0000:00:03.0'] to 'igb_uio'
[INFO] SETUP COMPLETE

Status - shows the interfaces under DPDK and Linux control. In the example below one interface is under DPDK control and the other (enp0s8) is under Linux control:

pcapplusplus@ubunu:~/PcapPlusPlus$ sudo python setup_dpdk.py status

Network devices using DPDK-compatible driver
============================================
0000:00:03.0 '82540EM Gigabit Ethernet Controller 100e' drv=igb_uio unused=e1000,vfio-pci,uio_pci_generic

Network devices using kernel driver
===================================
0000:00:08.0 '82540EM Gigabit Ethernet Controller 100e' if=enp0s8 drv=e1000 unused=igb_uio,vfio-pci,uio_pci_generic *Active*

No 'Baseband' devices detected
==============================

No 'Crypto' devices detected
============================

No 'Eventdev' devices detected
==============================

No 'Mempool' devices detected
=============================

No 'Compress' devices detected
==============================

No 'Misc (rawdev)' devices detected
===================================

This command take the following parameters:

Parameter Description
-v, --verbose Print more verbose output

Restore - restores the Linux configuration to its previous state before setup_dpdk.py setup was run. Please note that this command uses the setup_dpdk_settings.dat file to identify the previous state. If this file was deleted or moved the restore process will most likely fail. Please also note that a machine restart will probably restore most of this configuration, but this command enables restore without a machine restart.

If everything goes well the output should look something like this:

pcapplusplus@ubunu:~/PcapPlusPlus$ sudo python setup_dpdk.py restore
[INFO] removed hugepages
[INFO] bound device '0000:00:03.0' back to 'e1000'
[INFO] restored interface 'enp0s3'
[INFO] removed 'igb_uio' kernel module
[INFO] RESTORE COMPLETE

This command take the following parameters:

Parameter Description
-v, --verbose Print more verbose output

Initialization at application startup​

When using DPDK in your application it should be initialized on application startup. This configuration includes:

These steps are wrapped in one static method that should be called once in the application startup:

DpdkDeviceList::initDpdk();

If this methods succeeds DPDK is ready to be used in your application.

Next steps​

If you're curious to learn more, please refer to the following resources: