Making AWS News stupid fast with smart caching
As I’m typing this, Re:Invent 2024 is only weeks away. In anticipation of https://aws-news.com‘s busiest period of the year, I redesigned the API access patterns to support very effective caching. This resulted in significantly reduced backend load and a much faster frontend.
This article focuses on CDN caching – storing HTTP responses on points-of-presence (POPs) near the user. When you read ‘caching’ in this article, we’re talking about caching in CloudFront, unless stated otherwise.
My strong suit is backend development in Python on AWS. Conversely, I’m quite lacking in frontend and Typescript skills (although they’re getting better). So when I started building AWS News I naturally composed most of the business logic in the backend. This included enrichment features, such as mark as read, bookmark, and rating of articles. As a result, retrieving articles from the backend always yielded a personalized response. In this article we’ll learn why this approach does not promote efficient caching, and what the better alternatives are.
Notice how we’re combining the three types of data? Since this response contained personalized data, it was impossible to cache efficiently. Maybe the response could be cached for repeat visits by a single user, but as soon as the user data or popularity of the article changed, the cache would need to be invalidated. In practice, I did not cache these responses at all. Efficiency and cost aside, this meant the user generally had to wait for 300-700ms for the different data sources to be retrieved, combined, and sent over the wire. For every request. Not a great user experience.
