Why Every Engineering Team Should Embrace AWS Graviton4

Two years ago, we shared our experiences with adopting AWS Graviton3 and our enthusiasm for the future of AWS Graviton and Arm. Once again, we're privileged to share our experiences as a launch customer of the Amazon EC2 R8g instances powered by AWS Graviton4, the newest generation of AWS Graviton processors. This blog elaborates our Graviton4 preview results including detailed performance data. We've since scaled up our Graviton4 tests with no visible impact to our customers.

Our provisional measurements from internal dogfood testing six months ago said that we could "run 25% fewer replicas on the Graviton4-based R8g instances compared to Graviton3-based C7g, M7g, or R7g instances—and additionally 20% improvement in median latency and 10% improvement in 99th percentile latency"—but when it came time to reproduce with a larger sample size in production, we encountered problems where the numbers didn't look quite like what we saw in our early tests.

Within our own VPCs, the telemetry path from our production environment through Refinery into dogfood's Shepherd is very fast. We over-provision Refinery so there should never be a backlog of requests waiting to be sent from production to dogfood—but this is not at all a valid assumption for real-world clients of the production instance of Honeycomb. The effect of network buffering and slow-sending clients becomes noticeable as a fixed cost to latency and throughput. That's right: Graviton4 is so fast that what previously was a rounding error for receiving off the network and buffering the payload into memory cannot be eliminated no matter how fast the CPU gets.

Leave a Comment