tcmalloc's Temeraire: A Hugepage-Aware Allocator
Today’s paper is Beyond malloc efficiency to fleet efficiency: a hugepage-aware memory allocator by A.H. Hunter, Chris Kennelly, Paul Turner, Darryl Gove, Tipp Moseley, and Parthasarathy Ranganathan from OSDI ‘21.
This paper is a great read, less for its key results – a hugepage aware allocator that reduces tlb misses and memory usage – but for its meta-lessons: measure the entire system, invest in tooling, build a process to experiment confidently, profile globally act locally.
Of course the paper does actually contain lots of interesting tidbits about Temeraire, the hugepage aware memory allocator added to tcmalloc , so we’ll cover those too. But my key takeaways are all the higher-order lessons from the paper.
All of that sweet, sweet memory your programs are reading and writing to is virtual memory that is backed by your operating system by actual physical memory. Your processor and operating system work together to ensure that when a process accesses this virtual memory, it gets translated to an actual physical memory address, and these translations get cached in the translation lookaside buffer or TLB. This virtual memory is broken up into pages, historically 4 KiB in size, and the TLB caches the translations for a given virtual page to a physical page.
