My Top 3 Takeaways From Building Scalable Databases With DynamoDB

I spent the past 5 years building scalable databases for clients and organizations.

Here are a few lessons I’ve learned along the way.

A lot of these are issues a lot of DynamoDB users have and learning from these will set you up with an unfair advantage.

1. Understand your access?patterns

Before you start any database design with DynamoDB, stop and do this instead: identify and understand your data access patterns.

What I mean by this is to take some time to find out how your users will be reading and writing data on your application.

For example, are users browsing your products by category or rather by most popular? (it could be both)

Do you allow sellers to add multiple products at a time or rather just one?

By the manner your application fetches data most commonly that should tell you how you model it.

Rather than using a general data model, model your data (that is primary keys) based on how they will be queried.

If your application often queries users’ data along with their orders and purchase history, then you should store user info, orders and purchases using the same partition key.

DynamoDB optimizes for latency at scale and designing based on your application’s access patterns will make or break that speed and scalability.

2. Partition Key design lets you?scale

Always think high cardinality first.

High cardinality is simply a measure of uniqueness of your partition keys.

A high cardinality partition key is for example “userID” or “productID”?—?there is usually just one in your database.

A low cardinality partition key is for example “status” or “isFeatured”?—?there can be many shared values like “active” or “true/false”.

The higher the partition key cardinality the better your database can scale. Partition keys that have too many shared values will run into hot partitions?—?partitions that get too much traffic.

Your partition doesn’t always have to have a single unique value, sometimes you have no choice but to give it a value that will be shared. The thing to keep in mind here is when the partition key starts getting popular you should look towards sharding it.

Sharding partition keys can be as simple as adding a prefix before the partition key name, such as a date or a location value. That can further make the partition higher in cardinality.

3. Sort Key design lets you?filter

DynamoDB’s API has a “FilterExpression” method. This lets you filter items will high flexibility.?

Don’t use it.

Why??

Well because FilterExpressions will fetch all of the data in the query and apply the filter after having fetched the data. This essentially means you are spending the same capacity units and costs without the filter operation.

So how can you filter data in DynamoDB instead?

Use the sort keys.

Sort keys support the following query operations:

• = (equality)
• begins_with() (sort key starts with substring)?
• <= (less than or equal to)
• >= (greater than or equal to)
• BETWEEN (sort key is between value a and b)

Using particularly the begins_with() operator, we can perform some powerful filtering.

For example, if we need to filter hotel rooms by features we can use the following sort key design:

room#<view-type>#<num-of-guests>#<features>#<floor-number>#<room-number>

e.g.: 
Room 1: "room#sea-view#2-guests#smoking#f1#101"
Room 2: "room#sea-view#3-guests#smoking#f2#201"
Room 3: "room#garden-view#4-guests#no-smoking#f3#301"        

With the data model above we can filter out rooms by their view, number of guests, features (like smoking allowed), floor number and room number.

In this article, I go more into detail on using the begins_with() method to perform these powerful filtering strategies.

Conclusion

These are my top takeaways for anyone wishing to start with DynamoDB.

Mastering DynamoDB takes time, but understanding your access patterns, designing high-cardinality partition keys, and using sort keys effectively will get you ahead in the game.?

By applying these principles, you’ll be able to build more scalable, cost-efficient databases that perform well for real-world applications.

?? My name is Uriel Bitton and I hope you learned something in this edition of Excelling With DynamoDB.

?? You can share the article with your network to help others learn as well.

?? If you're looking for help with DynamoDB, let's have a quick chat here.

?? I hope to see you in the next week's edition!