Spy on your Kubernetes cluster with BPF

If you’ve been using Kubernetes for some time, you’ll know that the best thing you can do is to invite hundreds of thousands of your best acquaintances to run arbitrary commands on your cluster without adult supervision. At some point in the future, you might wonder what those people that you once knew are still doing on your cluster.

BPF (Berkeley Packet Filter) is a virtual machine inside the Linux Kernel that classifies events and triggers actions when it receives one of those events. It allows you to inject code into the kernel at runtime to handle those events; no kernel compilation required. There are two flavors of BPF. This article refers to the extended version, eBPF, but I just call it BPF.

So, getting back to those pesky friends hogging all the resources in your cluster. One of your options is to check all their pod definitions to determine the entry point for all their containers. This will only give you the first program that the container started. You’ll never know if their entry point spawned a million other processes that are actually mining the crypto currency of the week.

Another more interesting option is to use Kubectl Trace to check what’s running in a container. This extension for Kubectl schedules one-time-only jobs to run BPF programs inside a container. So you can list all processes running in any container with a one line command:

Leave a Comment