Irene Y. Zhang: The Case for a High-level Kernel-Bypass I/O Abstraction

submited by

Style Pass

2024-10-06 21:00:04

After chatting with people at HotOS, I was inspired to write this blog post summarizing my HotOS paper. At a high-level, the HotOS paper advocates for a new high-level I/O abstraction for kernel-bypass.

In the past, kernel-bypass has been associated with giving applications low-level access to devices. We argue that modern kernel-bypass applications should use a high-level abstraction instead, provided by a user-level, library OS that runs alongside the application in its address space.

In this blog post, I will motivate the problem: why we need a high-level interface for kernel-bypass I/O. The Demikernel OS architecture is a solution: it defines a high-level I/O abstraction for kernel-bypass and supports this abstraction across different kernel-bypass devices with library OSes. Read the HotOS paper if you are interested in our proposed solution.

Existing kernel-bypass abstractions (e.g., DPDK, RDMA) are low-level because they expose hardware features and limitations to applications. As a concrete example, we will look at sending data of arbitrary size and sending data at arbitrary speed with DPDK and RDMA. Note that both are easy with a high-level interface like POSIX sockets, which let applications send data of any size and at (almost) unlimited speed (until the OS runs out of memory for buffering).