Understanding DynamoDB Limitations For Better Database Design

Welcome to the 19th edition of Excelling With DynamoDB!

In this week’s issue, we'll discuss DynamoDB's limitations and understand some of the tradeoffs that must be taken when working with DynamoDB.

Limitations in any tool or technology are always important to understand, as they help us decide on tradeoffs and assess alternate possible solutions.

In DynamoDB these limitations affect how you model your data therefore understanding them is crucial to designing good databases.

Item Size?Limits

A single item in your DynamoDB table cannot contain more than 400KBs of data.

While this is sufficient for the great majority of use cases, it is smaller than what other database services offer.

• Firebase has a 1MB item limit
• MongoDB: 16MB item limit
• Couchbase: 20MB item limit
• ScyllaDB: 2GB item limit
• Cassandra: 2GB item limit

Why is this size limit smaller than most other databases?

The reason for the smaller size limit is that DynamoDB wants to protect you against high read and write latency.

Large items take longer to read and write from/to your database.

Items that are large in size also allow for fewer concurrent requests.

Query Size?Limits

When you query for multiple items in your database, DynamoDB will limit the query request to 1MB of data.

This is done again to deliver data at scale with the lowest latency possible.

If you have requests that exceed 1MB of data, you can simply paginate through the results by submitting a “follow-up” request.

Partition Throughput Limits

In DynamoDB, you have the choice to use provisioned throughput capacity or on-demand throughput capacity

• Provisioned capacity lets you define a range of throughput you wish for your database to support (a minimum and maximum capacity for DynamoDB to stay within)
• On-demand capacity mode lets DynamoDB dynamically provision throughput to match your traffic patterns

A read capacity unit is roughly 4KBs of data queried, a write capacity unit is 1KB of data queried.

A single partition?—?the storage node you define (with a partition key) for where to store your items?—?can have a maximum of 3000 read capacity units and 1000 write capacity units per second.

These limits are exceedingly generous as RCUs and WCUs work on a per-second basis and are applied to the partition level only, not the table as a whole.

Local Secondary Index?Limits

Finally, another important limit to be aware of is with regard to local secondary indexes (or LSIs).

An item collection?—?a group of items with the same partition key?—?cannot be larger than 10GB in an LSI.

This could become problematic since if you exceed this limit, your writes will get rejected.

To solve this limitation, you can consider using a global secondary index (GSI).

Another possible solution would be to shard your data further.

With GSIs however, there is virtually no limit on the size an item collection can grow to, although it is bad practice to make an item collection too large.

Conclusion

Like with all other databases and tools, DynamoDB comes with its limitations.

But in DynamoDB, these limitations are mostly intentional; designed that way to not let you design a bad query or keep its promise of a single-digit millisecond latency database.

Understanding these limitations in DynamoDB will help you design better and more efficient databases capable of massive scalability.

If you enjoyed this post, please consider subscribing and sharing the newsletter with your network: https://www.dhirubhai.net/newsletters/7181386750357893121/

?? My name is Uriel Bitton and I hope you learned something in this edition of Excelling With DynamoDB.

?? You can share the article with your network to help others learn.

?? If you enjoyed this post you can subscribe to my newsletter on Medium to get my latest articles on DynamoDB and serverless by email.

?? I hope to see you in the next week's edition!

References

This article was inspired and examples were taken from The DynamoDB Book by Alex Debrie. https://www.dynamodbbook.com/