Networking stacks are used in two ways: endpoints and forwarders.

Forwarders are usually not doing much with packets, just reading a few fields and choosing an output port. These are very common function in network cores (telcos, datacenters, ...).

DPDK is not well-suited for endpoint application programming, even though here you can still squeeze some additional performance.

But don't dismiss a framework that is currently so widely deployed just because you are not familiar with its most common use-case.

Sometimes you really don't do much more on packets than receive and store in (e.g) ring buffers until you can trigger some computation on aggregated data. Sometimes your application has critical latency and you need very light parsing. Not everyone uses tcp and all the options, some people are just down to jumbo udp packets, no fragmentation, and 800Gb/s rx...

It’s typically done when end to end latency is more important than full protocol compliance and hardening against myriad types of attackers. High frequency trading is typically where the applications where you see these sorts of implementations being really compelling.

For these types of applications there are proprietary implementations that you can buy from vendors that are more suited to latency sensitive applications.

The next level of optimization after kernel bypass is to build or buy FPGAs which implement the wire protocol + transport as an integrated circuit.

FPGAs are falling out of favor, ASICs are sufficient to implement most network processing.

Funnily, one of the biggest DPDK feature is an API to program smartNICs exactly in that way.

Fair enough. My Knowledge of what’s on the bleeding edge and mass adoption in algo trading is fairly dated. I haven’t been adjacent to that industry in over 5 years. Even back then there was turnkey ASICS you could buy that would implement the network plus whatever protocols you used to talk to the exchange.

I think the neatest thing that sticks out to me is how logging in implementations of low latency trading applications was essentially pushed to the network. Basically they just have copper or fiber taps between each node in their system and it gets sent to another box that aggregates the traffic and processes all the packets to provide trace level logging through their entire system. Even these have solutions you can buy from a number of vendors.

DPDK is common within network appliances, where you don't need most of the features within kernel networking stack.

That's not the reality. The kernel network stack does a lot of things that for some applications are not needed, so DPDK is used for those cases when they need very high performance. For example, packet capture, routing, packet generation... I don't think I've seen anyone rewriting what the kernel does precisely because when you use DPDK you don't want to do what the kernel does.

this is completely wrong. as the article points out, there are plenty of tcp stacks and other user space stacks available. dpdk is used in places where performance matters, and just because you haven't worked in that area doesn't mean it's useless. all telcos use it for the most part.

> They then spend the next few years writing their own stack

Supercomputing people have done this repeatedly and successfully.

A more general purpose stack is UDP-based applications.

I wouldn't do this with TCP, which I agree is complicated and difficult to get right.

Normally the endpoint stack you need is more limited initially and you don't have to have all the baggage. The biggest advantage you get at the end is zero copy network and if what you do involves tons of reading and writing to the network you can definitely gain performance in both bandwidth and latency by better using your CPU.

You can do XDP as well and gain zero copy but for a small subset of NICs and you still need to implement you own network stack I don't see ac way to avoid it much. There are existing TCP stacks for DPDK that you can use as well.

Apps probably don’t but networking related code might benefit a lot from DPDK.

How does this differ from eBPF?

eBPF pushes computation into the kernel to be colocated with kernel data/events. DPDK pushes all of the relevant data (the network packets) into user space to be colocated with user data/events.

You can mix both a bit to make the selection of where packets go with a eBPF program, and running higher level stacks in user space where user space is still dealing with raw packets rather than sockets.

And now you're in AF_XDP territory, which is an alternative to dpdk, a tentative to normalise kernel 'bypass' for network operations.

DPDK has an XDP driver too so you can use that and also work with network cards that do not support XDP or the zero copy variant for more flexibility. Though DPDK can be a pain in and of itself, it is fairly opinionated and I don't like all of its opinions. It has its roots in switching and routing applications and using it for an endpoint can have some mismatches in opinion. It is open source so you can modify it which works out for me in the end.

I wouldn't say XDP is an alternative to DPDK. XDP severely limits what you can do as you insert code inside of the kernel. DPDK exposes the NIC to regular user code so you can do what you want. XDP is useful when you want to modify what the kernel does with packets before they go to their destination application, DPDK when you want to do whatever with the packets without sending them to any other application.

That's why the parent mentions AF_XDP. AF_XDP allows to get raw packets into userspace, bypassing the kernel stack. There users can run e.g. a custom TCP stack to process the packets. That's similar to what can be done with DPDK.

And both tools allow to export a "regular NIC" to the user. With AF_XDP one would probably just run the AF_XDP socket in a way which intercepts packets for a certain port, and leave all other traffic alone. That would then flow through the regular kernel stack, and the network interface would continue to show up as without XDP.