Caching secrets of the HTTP elders, part 1
There is no hard limit on table size in csvbase. Within reason, they can be as big as you like. And if you go off and run your own instance you can do whatever you want. Big Data isn't always Better Data, but it's nice to have the option.
That said, wouldn't it be nice to have those tables cached, so that if you read it twice you don't have to fully download it again the second time?
Hang on though. If someone changes that table you'd be stuck looking at the old version for up to 24 hours. Being eventually consistent was once very trendy. But it's no longer hip - even Amazon S3 no longer returns stale data. So serving stale data just won't do for csvbase.
The elden ones thought deeply about caching. DNS is a cache. Virtual memory is a cache. And if you've ever wondered why HTTP/1.0 got a patch release (HTTP/1.1) less than a year after it came out: more caching.
HTTP/1.0 had almost nothing in the way of caching. In HTTP/1.1, a whole new top-level section on the subject was added to the standard. This was essential because, back in the day, the repeated re-downloading of unchanged files was a real headache.
