Efficient packet processing system based on dpdk

One 、 Concept

Intel DPDK Full name Intel Data Plane Development Kit, yes intel Data plane development toolset provided , by Intel architecture（IA） Efficient packet processing in user space under processor architecture provides support for library functions and drivers , It is different from Linux The purpose of the system is universal design , It focuses on the high-performance processing of data packets in network applications . It has been verified that it can run on most of the Linux On the operating system .DPDK Used BSDLicense, It is very convenient for enterprises to realize their own protocol stack or application based on it . There are a lot of things that are based on dpdk High performance network framework for ,OVS and VPP Is a commonly used data plane framework ,mTCP and f-stack It is a common user mode protocol stack .

It's important to note that ,DPDK The application is running in user space, using its own data plane library to send and receive packets , Around Linux The kernel protocol stack processes packets .Linux The kernel will DPDK An application is seen as a normal user mode process , Including its compilation 、 Linking and loading are no different from normal programs . Here's the picture 2 Shown DPDK The package processing flow bypasses the kernel and goes directly to the user layer for processing , It is different from the traditional data packets that go to the kernel first and then to the user layer .

Kni(Kernel NIC Interface) Kernel NIC interface , yes DPDK Solution that allows user mode and kernel mode to exchange messages , for example DPDK The protocol stack is dedicated to DNS message , The rest of the messages pass through KNI Interface is returned to the kernel for processing .KNI Simulated a virtual network port , Provide dpdk Applications and linux Communication between kernels . be used for DPDK Interaction with the kernel ,kni The interface allows messages to be received from the user state and forwarded to the kernel protocol stack .DPDK All of the packages in the user space use memory pool management , The memory interaction between kernel space and user space does not need to be copied , Control transfer only .DPDK The main external function interfaces of are all in rte_ As a prefix , Abstract function interface is a typical software design idea ,rte Refer to runtime environment,eal Refer to environmentabstraction layer. The figure below 3 yes kni Of mbuf Use flow chart , You can see the flow direction of the message , Because the message in the code is actually a memory pointer . among rx_q On the right is user mode , On the left is the kernel state . Finally, by calling netif_rx() Send a message to linux Protocol stack , In this, we need to put dpdk Of mbuf convert to skb_buf. When linux towards kni When the port sends a message , Call callback function kni_net_tx(), Then the message is converted and sent to the port .rte_pktmbut_free() Release memory back to mbuf In the memory pool .

Two 、DPDK Of Helloworld Code example

HelloWorld Is the most basic introductory program , The code is short , It's not complicated . It builds a multi-core （ Threads ） The basic environment of operation , Each thread prints “hello from core #”,core # Is managed by the operating system .

int  main(int    argc, char **argv) 
{
     
   int ret; 
   unsigned lcore_id; 
   ret = rte_eal_init(argc, argv);
 
   if (ret < 0) 
          rte_panic("Cannot init EAL\n");
  
   /* call lcore_hello() on every slave    lcore */ 
   RTE_LCORE_FOREACH_SLAVE(lcore_id)    {
     
          rte_eal_remote_launch(lcore_hello,    NULL, lcore_id); 
   } 
 
   /* call it on master lcore too */ 
   lcore_hello(NULL); 
   rte_eal_mp_wait_lcore(); 
   return 0; 
}

about HelloWorld The instance , The most required parameters are “-c ”, Thread mask （coremask） Specifies the threads that need to participate in running （ nucleus ） aggregate .rte_eal_init The work done by itself is complex , It reads the entry parameters , Parse and save as DPDK Operating system information , Rely on this information , Build a runtime environment designed for package processing . The main actions are divided into ： Configuration initialization –> Memory initialization –> Memory pool initialization –> Queue initialization –> Alarm initialization –> Interrupt initialization –>PCI initialization –> Timer initialization –> Detect memory localization （NUMA）–> Plug in initialization –> Main thread initialization –> Polling device initialization –> Establish a master-slave thread channel –> Set the slave thread in wait mode –>PCI Device detection and initialization … about DPDK Users of the library , These operations have been EAL encapsulated , Clear interface . If necessary DPDK Deep customization , Secondary development , Internal operations need to be carefully studied , See the official website for details https://doc.dpdk.org/guides/prog_guide/.

DPDK Multi core oriented design , The program will try to run exclusively on the logical core (lcore) On .Main The important part of the function is to start the multi-core running environment ,RTE_LCORE_FOREACH_SLAVE（lcore_id） As the name suggests , Traverse all of EAL Specify available lcore, And then through rte_eal_remote_launch At every lcore On , Start the specified thread .

int rte_eal_remote_launch(int (*f)(void *),
void *arg, unsignedslave_id);

The first parameter is from the thread , It's the thread being called up , The second parameter is the parameter passed to the slave thread , The third parameter is the specified logical core , The slave thread will execute in this core On . say concretely ,int rte_eal_remote_launch(lcore_hello,NULL, lcore_id); Parameters lcore_id Specifies the slave thread ID, Run the entry function lcore_hello.
Operation function lcore_hello, It reads its own logical core number (lcore_id), Print out “hellofrom core #”

static    int 
lcore_hello(__attribute__((unused))    void *arg) 
{
    
 
   unsigned lcore_id;
 
   lcore_id = rte_lcore_id();
 
   printf("hello from core %u\n", lcore_id);
 
   return 0;
 
}

The above is just a simple example , In real DPDK Deal with... In the scene , The handler function will be a loop running process .

DPDK It also has its own disadvantages , For low load scenarios, it is not recommended to use DPDK：

The transfer of the kernel stack to the user layer increases the development cost .
Low load servers are not practical , Will cause the kernel to idle .

Study address ：https://ke.qq.com/course/5066203?flowToken=1043068
B Station Teaching ：https://www.bilibili.com/video/BV1Ju411z773?spm_id_from=333.999.0.0

DPDK Systematic learning materials 、 Instructional video and learning roadmap , Free sharing, you can add your own learning exchange group if you need it 973961276 obtain