Skip to main content

Leaderless replication

·2 mins
  • I wrote yesterday that last-write-wins is a poor idea because it loses data (so, no durability). However, that’s okay when the data is immutable.
  • A major issue to worry about is how to manage conflicts.
    • Remember:
      • There is no order of writes. Also, if there aren’t rollbacks on partially failed writes, concurrent writers can write in different order on the nodes.
      • Also, applicable to multi-leader replication.
    • What’s causality or concurrent writes?
      • Can’t think in terms of time. Instead, think in terms of dependencies. If one update to a record builds on top of another, they are causally connected and not concurrent.
      • So, how do we detect this dependency or causality? Versioning, as described below, helps.
    • Apart from CRDTs, version vectors are useful.
      • Note: Check this for more details.
      • Best way to understand them is to consider one replica and one version number per record. When you extend that idea to multiple replicas, you need to use a vector/array of version numbers.
      • Writer: read from replica, merge its own data with what it received and send it back for writing. It also gets a version number during the read and needs to send the same back while writing.
        • Client shouldn’t be expected to modify version numbers, like incrementing by 1, while writing back: that’s the job/concern of the database.
        • During the read, the client can receive multiple versions. Hence, the need to merge.
      • When replica receives a write, it throws away all versions that are equal or lesser than the one it got because they are assumed to be merged.
    • Seems complicated because application developers need to write business logic for merge. Some DBs also allow you to supply small piece of code which the DB will automatically run during merges.