Sharding or Partitioning

Sharding or Partitioning

Technique used by some data stores to reduce contention without sacrificing consistency.

Records are distributed across nodes using a Shard Key or a Partition Key that will be used by the Database Router that redirects requests to the correct shard/partition.

Benefits:

  • Contention is isolated to a shard/partition.
  • Given that each shard stores a part of the dataset, it is only handling a small part of the overall load.
  • Improves elasticity.

However, with time the database will still be a bottleneck because there may be too many users acesssing the shards/partitions (sometimes it may be a bad choice of partition key).