What Is Split For Heat In DynamoDB?

It is well known that Amazon DynamoDB adapts to workloads of any scale while offering consistent performance.

But a lesser-known concept with its adaptive capacity features is “split for heat”.

To understand what “split for heat” is we must first understand the reason behind it.

In DynamoDB, your data is written to partitions on AWS’s storage servers.

While a partition has a physical size limit, you as a customer, are not concerned with that nor will it affect you in any way. This is because DynamoDB will automatically manage partitions for you, creating new ones as your dataset grows.

However, even though size limits is not an issue, hot partitions surely is one.

A hot partition is a partition that is consistently receiving high traffic. This can impact read and write performance to and from that partition (when reading or writing items stored with that partition key).

The standard prevention for a hot partition is high partition key cardinality.

But a hot partition can still occur when you have an overly popular partition.

This is where DynamoDB applies a strategy called “split for heat”.

What is Split For?Heat?

Split for heat is a mechanism where DynamoDB detects a partition that is receiving high traffic and automatically splits it into two smaller partitions.

This helps reduce throttling and alleviates the performance impacts of reads and writes to that partition as you now have double the throughput available for that data (since it’s split into two).

This is better than traditional sharding as it requires no manual intervention. DynamoDB will also determine when it is best to split it based on usage patterns.

When splitting a partition, DynamoDB will distribute the items based on their sort key. This effectively doubles the available read and write throughput.

How Does Split For Heat?Work?

1. DynamoDB will monitor for read and write traffic to its partitions. When a partition is identified as receiving sustained high traffic, it is marked as a candidate for splitting.
2. The partition is split into two, with items distributed according to their sort key. Each new partition gets around half of the original items.
3. Once the partition has been split, the total read/write capacity doubles, preventing throttling
4. While DynamoDB provides no notification when a split occurs, users will notice the reduction in throttling and better overall request handling.

Best Practices to maximize the benefits of Split For?Heat

Some best practices and design considerations:

• Design partition keys carefully: even though DynamoDB can adapt to hot partitions, it’s always good to choose high cardinality partition keys to distribute writes evenly.
• Avoid time-based sort keys for high traffic items: if items in a partition have an ever-increasing sort key (e.g. timestamps), they will always be directed to a single partition, making the split for heat ineffective. Consider a more randomized sort key design in this case.
• Monitor and adjust: use Amazon CloudWatch to track read and write capacity usage and identify hot partitions before they impact your system’s performance.

Conclusion

Split for heat is one of DynamoDB’s most valuable features for managing unpredictable traffic use cases.

While proper data modeling is crucial to prevent hot partitions, DynamoDB’s adaptive capacity features like split for heat make sure that even the highest traffic workloads don’t impact your system’s performance.

?? My name is Uriel Bitton and I hope you learned something in this edition of Excelling With DynamoDB.

?? You can share the article with your network to help others learn as well.

?? If you're looking for help with DynamoDB, let's have a quick chat here.

?? I hope to see you in the next week's edition!