In a distributed system, nodes need to be able to tell which value for a key is the most recent.       Sometimes they need to know past values s

Versioned Value

submited by
Style Pass
2021-06-23 13:00:07

In a distributed system, nodes need to be able to tell which value for a key is the most recent. Sometimes they need to know past values so they can react properly to changes in a value

Store a version number with each value. The version number is incremented for every update. This allows every update to be converted to new write without blocking a read. Clients can read historical values at a specific version number.

Consider a simple example of a replicated key value store. The leader of the cluster handles all the writes to the key value store. It saves the write requests in Write-Ahead Log. The Write Ahead Log is replicated using Leader and Followers. The Leader applies entries from the Write Ahead Log at High-Water Mark to the key value store. This is a standard replication method called as [state-machine-replication]. Most data systems backed by consensus algorithm like Raft are implemented this way. In this case, the key value store keeps an integer version counter. It increments the version counter every time the key value write command is applied from the Write Ahead Log. It then constructs the new key with the incremented version counter. This way no existing value is updated, but every write request keeps on appending new values to the backing store.

Embedded data stores like [rocksdb] or [boltdb] are commonly used as storage layers of databases. In these storages, all data is logically arranged in sorted order of keys, very similar to the implementation shown here. Because these storages use byte array based keys and values, it's important to have the order maintained when the keys are serialized to byte arrays.

Leave a Comment