Downloading large files is hard.  Networks are messy, and while there are plenty of protocols that aim to abstract away this messiness, they all have

Reliable Immutable Transfer Protocol — binarycat

submited by
Style Pass
2024-11-07 21:00:36

Downloading large files is hard. Networks are messy, and while there are plenty of protocols that aim to abstract away this messiness, they all have their own problems.

If implemented properly, HTTP can handle essentially anything you throw at it... however, to my knowledge, there is no full http implementation that correctly implements every corner-case of error handling. (wget is probably the closest)

The main difficulty of HTTP is that it's too flexible for its own good: all the error recovery and detection mechanisms are optional headers, so a complete implementation has to have several layers of fallbacks, and testing all these rarely used code paths is a significant challenge. RITP addresses this complexity by limiting its scope and choosing a small number of mandatory fields that all servers must support.

One problem with http is that of mid-air collisions: it is entirely possible for a server to implement Range but not provide a Last-Modified or ETag field. RITP solves this by not allowing files/resources to be modified.

Leave a Comment