Why does GOMEMLIMIT take up significant physical memory for unused virtual memory?
While debugging memory bloat in a Go application recently, I found that removing the GOMEMLIMIT soft memory limit and disabling transparent huge pages partially mitigated the issue. However, I couldn't fully explain why these changes worked. So I thought why not ask the internet about it.
The following Go program vm-demo.go demonstrates memory bloat by allocating a 400 MiB slice every second. It saves references to the slices without any read or write operations.
I build this program with default go build settings (no additional flags). To monitor its memory usage, I collect virtual memory size (VmSize) and physical memory size (VmRSS) every second from /proc/${PID}/status, then plot these measurements over time. Use the collect.sh and plot-memory-usage.py to reproduce.
On Linux, virtual memory to physical memory mapping is handled cooperatively by the kernel (through page fault handling) and hardware (through MMU and TLB). Importantly, virtual memory doesn't always require corresponding physical memory allocation. That's how the GC ballast tricked the Go runtime into garbage collection based on virtual memory thresholds without consuming actual physical memory. This behavior is clear in Figure 1, where virtual memory gradually increases to 80GiB while physical memory only grows to about 90MiB.
