Replication, Sharding & Consistent Hashing · TL;DR

1 min read
Senior12 min read
Rapid overview

TL;DR

Replication scales reads and buys availability; it does nothing for write throughput, because every write still goes to one primary. Sharding is the only thing that scales writes, and it is expensive: you lose cross-shard joins and transactions, and you inherit the shard-key decision, which is effectively permanent and is where these systems go wrong. Consistent hashing exists because the obvious approach — hash modulo the node count — remaps almost every key when the node count changes, which is catastrophic for a cache and worse for a database.

See also