Sharding & partitioning Study Portal

A: Everything that scales without changing the data model: tune queries and indexes, add read replicas and a cache for read load, move large blobs to object storage, archive or partition cold data by time within the same database, scale the primary vertically (modern single nodes handle terabytes and tens of thousands of writes per second), and split by function — separate databases per service or bounded context. Shard when write throughput or dataset size genuinely exceeds a single primary with headroom, because sharding makes joins, transactions, uniqueness and operations permanently harder.

A: New writes always carry the newest timestamp, so they all fall into the last range and one shard takes the entire write load while the others sit idle on historical data. Avoid it by prefixing the key with something that spreads writes — a hash of the device or user id, or a small bucket number — so writes for the same moment land on many shards, and use a compound key such as (sensor id, timestamp) so per-sensor time ranges are still contiguous. The trade-off is that a query for "all events in the last minute" must now read every shard.

A: Because the modulus changes the shard of almost every key: a key maps to a new shard unless hash mod N and hash mod N+1 happen to agree, which for going from 4 to 5 shards is only about one key in five. So roughly 80% of the data has to be copied across the network, while the cluster is under the load that prompted the change. Consistent hashing or a fixed number of virtual partitions assigned to nodes limits movement to about 1/N of the data.

A: Uneven load and uneven failover. With one ring position per server, random placement leaves some servers owning much larger arcs than others, and when a server dies its entire arc moves to its single clockwise neighbour, possibly overloading it. Giving each server many virtual positions averages the arcs out, spreads a failed server's keys across all the remaining servers, and lets heterogeneous hardware take a proportional number of positions.

A: Tenant id is usually the best choice: nearly every query is already scoped to one tenant, so it routes to a single shard; a tenant's data stays together, so joins and transactions remain local; and moving or isolating a tenant is a shard-level operation. The weakness is skew — one enterprise tenant can be thousands of times larger than the median, overloading its shard. Mitigations: place large tenants on dedicated shards via a directory, or use a compound key (tenant id plus a sub-key) for the few tenants that exceed a single shard.

A: Split the counter. Store it as N sub-counters (for example 50 keys, likes:product:0 to likes:product:49) that hash to different shards; each increment goes to a random sub-counter, spreading writes across shards, and a read sums the N values — cached for a second or two, since exact real-time accuracy is not needed. Alternatively, buffer increments in memory or a stream and apply them in batches. The price is a slightly more expensive and briefly stale read.

A: Copy a snapshot of the moving range to the destination, catch up with a change stream (CDC or the replication log) until the destination lags by seconds, briefly block or queue writes for that range while the last changes apply, flip the routing map so clients go to the destination, then verify counts and checksums and delete from the source after a safety period. The risk concentrates at the cutover: a write accepted by the source after the final catch-up but before the routing flip is lost unless writes are fenced, and clients with a cached old routing map keep writing to the source — so the source must reject writes for ranges it no longer owns.

A: A local index lives on each shard and covers only that shard's rows, so it is updated in the same local write and is consistent, but a query by the indexed attribute must ask every shard (scatter-gather). A global index is itself partitioned by the indexed value, so a query goes to one index partition, but each base write must also update an index entry that may live on another shard — usually asynchronously, so the index can briefly lag the data. DynamoDB exposes exactly this choice as LSI versus GSI.

A: The uniqueness check needs a single place keyed by email. Keep a separate lookup table sharded by email (email to user id) and claim the email there with a conditional insert before, or transactionally with, creating the user; if the claim fails the email is taken. If user creation fails after the claim, release it or let a cleanup job reconcile orphaned claims. Scatter-checking every user shard is racy — two sign-ups can both see no match — so it does not enforce uniqueness.

A: Random UUIDs insert at random positions in the primary key B-tree, causing page splits, poor cache locality and write amplification as the index grows beyond memory. Time-ordered ids append near the end of the index like an auto-increment, keeping the hot pages in cache, while still being generated without coordination on any node. They are also roughly sortable by creation time. The trade-off is that they leak approximate creation time, and in a range-partitioned store they concentrate new writes, so they suit hash-partitioned or single-node indexes best.