AWS: DynamoDB

DynamoDB is a serverless NoSQL distributed database which scales horizontally. And the data inside DynamoDB is stored in multiple AZs.?

DynamoDB guarantees single digit latency ( Only for good database designs though?). DynamoDB can scale to MILLIONS of requests per second.

DynamoDB Characteristics:

• A row in DynamoDB table is called an Item.
• Maximum Size of an item is 400KB.
• Data Type Supported are:

1. Scalar Type: String, Number, Binary, Boolean, Null
2. Document Type: List, Map
3. Set Types: String Set, Number Set, Binary Set

• A DynamoDB Table has a primary key. You can define primary key in two ways:

1. Primary Key can be Partition/HASH key.
2. Primary Key can be a combination of Partition/HASH key and Sort/RANGE key.

• You can only define primary key while creating the table and no other columns. Rest of the columns are automatically added when you insert an Item in table.

Documentation:?https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Introduction.html

POM Dependency:

<dependency> <groupId></groupId> <artifactId>aws-sdk-factory</artifactId> <version>2.0.2.AWSNAT-LIBRARY-42</version> </dependency>

1) How data is stored in DynamoDB

Data is stored in partitions inside DynamoDB.?

1) In case of Primary Key as Partition/HASH Key:

2) In case of Primary Key as a combination of Partition/HASH and Sort/RANGE key:

2) Local Secondary Index and Global Secondary Index

• Local Secondary Index (LSI): While creating a local secondary index we can choose a different sort/RANGE key than the primary index but Partition/HASH key must be same as that of primary index. This means that we can create an index in which Partition/HASH key will be same as that of primary index but we can change Sort/RANGE key thus changing the order in which items are arranged inside each partition. Primary Index is not affected when creating LSI. Documentation:?https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/LSI.html
• Global Secandary Index (GSI): While creating a global secondary index we can choose a different partition/HASH as well as sort/RANGE key than that of primary index. GSIs are helpful when we have use case in which we are not able to query primary or other secondary indices. (We can still use SCAN operation on primary index and filter the data inside JVM; But that is not recommended).? You can think of GSI as a completely new table. Documentation:?https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/GSI.html

Whenever we write to a table, the item is written to primary index. The changes are propagated to LSIs and GSIs of the table. We don't have to write to them explicitly.

We can query LSIs and GSIs directly!

Note:

• There is a limit on how many LSIs and GSIs can a table contain.
• Reading from LSI can be strongly or eventually consistent ( depend on you). But reading from GSI will always be eventually consistent.

3) Write Capacity Units and Read Capacity Units

We can configure our table to scale on demand. In that case our DynamoDB table can handle many number of requests without throttling. Otherwise, we can configure read and write capacity units explicitly and if the load goes beyond that measure. DynamoDB starts throttling and gives exception as response.

Documentation:?https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.ReadWriteCapacityMode.html

4) Query Examples

Table Structure:

1) Primary Index:

ProfileIdConsumerId as Partition/HASH Key

InvoiceId as Sort/RANGE Key

2) Global Secondary Index (StatusIndex):

Status as Partition/HASH Key

CreatedDateInvoiceId as Sort/RANGE Key

• QuerySpec to fetch all the invoices of one particular profile: KeyConditionExpression?→?ProfileIdConsumerId = :ProfileIdConsumerId ValueMap?→?{:ProfileIdConsumerId=DataCorrectionTestSAT4|VGI-US-DIGITAL_ADVICE_TEST} The above query spec will fetch all the invoices of?DataCorrectionTestSAT4|VGI-US-DIGITAL_ADVICE_TEST this partitionKey.

• QuerySpec to fetch all the invoices of one particular profile having status in the list provided: KeyConditionExpression?→?ProfileIdConsumerId = :ProfileIdConsumerId FilterExpression?→?#st?IN (:st1)

ValueMap?→ {:st1=SETTLEMENT_PENDING, :ProfileIdConsumerId=DataCorrectionTestSAT4|VGI-US-DIGITAL_ADVICE_TEST} NameMap?→?{#st=Status} The above query spec will fetch all the invoices of?DataCorrectionTestSAT4|VGI-US-DIGITAL_ADVICE_TEST?this partitionKey having status?SETTLEMENT_PENDING.

• QuerySpec to fetch all the invoices of one particular status (We will have to query StatusIndex GSI): KeyConditionExpression?→?#st = :st FilterExpression?→?ConsumerId = :ConsumerId
• ValueMap?→?{:st=SETTLEMENT_PENDING, :ConsumerId=VGI-US-DIGITAL_ADVICE_TEST} NameMap?→?{#st=Status} The above query spec will fetch all the invoices?having Status?SETTLEMENT_PENDING and ConsumerId as?VGI-US-DIGITAL_ADVICE_TEST.

• QuerySpec to fetch all the invoices of one particular status which was created between the dates provided (We will have to query StatusIndex GSI):
• KeyConditionExpression?→?#st = :st and CreatedDateInvoiceId between :startDate and :endDate FilterExpression?→?ConsumerId = :ConsumerId
• ValueMap?→?{:st=SETTLEMENT_PENDING, :ConsumerId=VGI-US-DIGITAL_ADVICE_TEST, :startDate=2020-03-27 00:00:00.000, :endDate=2020-03-26 23:59:59.999} NameMap?→?{#st=Status}
• The above query spec will fetch all the invoice having Status?SETTLEMENT_PENDING?and ConsumerId as?VGI-US-DIGITAL_ADVICE_TEST and also having CreatedDate between (2020-03-27 00:00:00.000 , 2020-03-26 23:59:59.999).

• UpdateItemSpec to update a particular invoice (We need to explicitly pass partition key and sort key while creating UpdateItemSpec):
• UpdateExpression?→?set LastUpdatedDate = :lastUpdatedDate, Invoice.lastUpdatedDate = :lastUpdatedDate, #st = :st, Invoice.#invoicest = :st, Invoice.daysInService = :daysInService, Invoice.grossFeeAmount = :grossFeeAmount, Invoice.netFeeAmount = :netFeeAmount, Invoice.expenseRatioFeeCredit = :expenseRatioFeeCredit, Invoice.transactionIds = :transactionIds, Version = Version + :Version
• ValueMap?→?{:lastUpdatedDate=2020-06-19 13:16:56.922, :st=STATEMENT_PENDING, :daysInService=32, :grossFeeAmount=145, :netFeeAmount=21.1243, :expenseRatioFeeCredit=141.5, :transactionIds=[123], :Version=1} NameMap?→?{#st=Status, #invoicest=status} The above Item spec will update the item having a particular partitionKey and sortKey that is explicitly passed while creating the spec.

QuerySpec Code:

QuerySpec querySpec = new QuerySpec() ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?.withKeyConditionExpression(String.valueOf(keyCondition)) ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?.withFilterExpression(filterExpression) ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?.withValueMap(valueMap) ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?.withNameMap(nameMap);

UpdateItemSpec Code:

UpdateItemSpec updateItemSpec = new UpdateItemSpec().withPrimaryKey(profileIdConsumerIdColumn, profileIdConsumerId, invoiceIdColumn, invoiceId) ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? .withUpdateExpression(String.valueOf(updateExpression)) ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? .withNameMap(nameMap) ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? .withValueMap(valueMap) ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? .withReturnValues(ReturnValue.ALL_NEW);

Thanks for reading the article ..!