Sharding & Hash Slots

A single Redis instance is incredibly fast, but it is bound by the physical limits of a single server’s RAM and CPU. To scale beyond a few hundred gigabytes or to handle millions of operations per second, you need to distribute data across multiple servers. This is called Sharding.

Redis Cluster uses a unique approach to sharding called Hash Slots.

1. The 16,384 Slots Concept

Instead of using consistent hashing (like Cassandra or DynamoDB), Redis divides the key space into exactly 16,384 buckets called Hash Slots.

Every key in your database belongs to one of these slots. The slots are then divided among the available Master Nodes.

Why 16,384?

This number wasn’t chosen randomly. It is 214. The reasoning involves a trade-off between:

  1. Granularity: More slots mean easier rebalancing.
  2. Message Size: The slot configuration is exchanged via the heartbeat packets (Gossip). A bitmap of 16k slots takes only 2KB of data, which fits easily in a standard MTU packet without causing network congestion.

2. The Algorithm: CRC16

To determine which slot a key belongs to, Redis uses the CRC16 algorithm modulo 16384.

Hash Slot = CRC16(key) % 16384

This operation is deterministic. The key user:100 will always hash to the same slot, regardless of how many nodes are in the cluster.

[!NOTE] This deterministic mapping is why Redis clients can cache the slot map. They don’t need to ask the server “Where is key X?” every time; they can calculate “Key X is in Slot Y” locally.

3. Interactive: Key-to-Slot Visualizer

Type a key below to see how it is hashed and assigned to a hypothetical 3-node cluster.

Key
user:100
→
CRC16
...
→
% 16384
...
Node A
Slots 0-5460
Node B
Slots 5461-10922
Node C
Slots 10923-16383

4. Hash Tags: Controlling Locality

Sometimes, you need multiple keys to live on the same node. For example, you might want to perform a transaction involving a user’s profile and their order history.

In Redis Cluster, transactions (MULTI/EXEC) and Lua scripts only work if all keys involved belong to the same hash slot.

To force keys into the same slot, you use Hash Tags.

  • If a key contains {...}, only the part inside the braces is hashed.
  • Example:
  • user:1000:profile → Hashes user:1000:profile
  • {user:1000}:profile → Hashes user:1000
  • {user:1000}:orders → Hashes user:1000

Because the string inside the braces is identical, both keys map to the same slot, guaranteeing they reside on the same node.

[!WARNING] Use hash tags sparingly. Overusing them can lead to Hot Shards, where a single node receives unevenly high traffic because too many keys are forced onto it.

5. Summary

  • Sharding: Splitting data to scale horizontally.
  • Hash Slots: 16,384 logical buckets.
  • CRC16: The deterministic algorithm used to map keys to slots.
  • Hash Tags: A mechanism to enforce data locality for multi-key operations.

In the next chapter, we’ll look at how these nodes talk to each other to form a cohesive cluster.