This is from a rather interesting session I had last year during AWS Loft Stockholm 2018 regarding DynamoDB as a NoSQL solution for Applications and the importance of analysing our application data access patterns in order to leverage DynamoDB or similar tools.

The speaker Rick Houlihan made the funny comment, that out there word is that NoSQL databases are super flexible, perhaps due to the absence of schema, but he thought they were the contrary, quite inflexible due to most of the time structures and data access operations were design and created for specific queries.

Food for thought, the reality is among the key factors for fast data access is having denormalized data and the right indexing strategy, these constructs can indeed become specific to a single or limited number of operations.

Features

DynamoDB has the following features:

• Fast and consistent.
• Fully Managed
• Key/Value or Documents as storage items.
• Highly scalable to any workload.
• Access control
• Suitable for Event-Driven Programming.

Now, down to the storage unit in DynamoDB, we find a Table which consist of the following elements:

Partition Keys

They uniquely identify an item, therefore they are mandatory. This key also drives the data disitribution across partitions, a good strategy and design for this key impacts directly scaling for data access.

They are ultimately built as an unordereded hash index.

Sort Keys

Sort keys are optional and more than one can be defined, they enable rich querying capabilities for items, such as:

• begins with, in, contains, between type of operations.
• Counts.
• Top/Bottom N values.
• Sorted result.
• Logical/Arithmetical operators.

Indexes

Local Secondary Index (LSI)

They provide the means to establish efficient data access aside from the table’s partition keys but rather by an alternate sort key.

Features:

• Index is local to a partition key
• Alternate sort key attribute.
• You can project some of the attributes in the table.

Conditions

• The partiton key is the same as that of its base table.
• The sort key consists of exactly one scalar attribute.
• The sort key of the base table is projected into the index.

Global Secondary Index (GSI)

In escence, they achieve the same goal as LSI with, a slight difference is that unlike LSI, GSI give the posibility to use several keys instead of just one.

Features:

• Index is across all parition keys.
• Uses composite sort keys for compound indexes

DynamoDB will take care of updating the GSI if needed as data changes in the main table, this process happens asynchronously. Something to look out for is the GSI write capacity, if it is reached, operationd on the tables wil be throttled.

LSI or GSI

In short, LSIs provide the means to have alternate sort keys based on the table’s partition keys, one of the keys, while GSIs can declare its partition key outside of those keys from the base table.

Some other considerations:

• LSI can me modeled as GSI.
• If the data size in a collection is greather than 10 GB, use GSI.

The aforementioned size limitation is due to the maximum size allowed of item collections, refer to the LSI documentation from AWS, particularly the section for Item Collection and size limits.

Scaling

This is achieved through partioning and how data is distributed across them; throughput for write and reads are measured by WCU and RCUs accordingly, something to remark is that RCU targets consistent reads specifically and eventually consistent reads cost half a RCU. Autoscaling is another interesting feature which provides adaptive throughput based on the traffic we are having.

Quoting the DynamoDB Developer Guide

To get the most out of DynamoDB throughput, create tables where the partition key element has a large number of distinct values, and such are requested fairly uniformly, as randomly as possible.

Data Modeling

In the past I have seen the use of NoSQL Databases where Team try to move an existing relational schema and data or start a fresh one but still thinking about a relational model, I have always imagined a NoSQL solution as a building piece in the bigger picture, there is nothing wrong with multiple stores if each has its own purpose, if you need your data shaped in more than way.

With NoSQL, it is all about aggregates and specific queries you need, you model your data and indexes accordingly, this is, for the most part, the secret sauce for optimal and speedy reads, data is exactly as you need it, no overhead gluing things together and applying any sort of transforms.

In the case of DynamoDB, the tenets for data modeling:

• Understand the use case.
  • OLAP or OLTP Application?.
  • ER Model.
  • Data Life Cycle Requirements.
    • TTL, Backup/Archival, etc.
• Identify the access pattern.
  • Read/Write workloads.
    • ID Data Sources.
    • Define Query aggregations.
    • Identity and document worklows.
  • Query dimensions and aggregations.
• Use of NoSQL design patterns.
  • Identify primary keys
  • Table per service (avoid unnecessary queries, simplify access patterns)
  • Indexes for secondary access patterns.

Some relationship examples with a data modeling approach:

• 1:1, Table or GSI with an alternate Partition Key, use the GetItem or BatchGetItem API methods.
• 1:N, Table or GSI with a Partition and Sort Key, use the Query API method.
• N:M, Table and GSI with Partition and Sort Key elements switched, use the Query API method, it is the GSI who’d give the inverse access query.

[//]: # Follow up with Design Patterns and Best Practices and beyond in part two.

References

• DynamoDB Partitioning and Replication
• DynamoDB: A Highly-Available Key-Value Store
• Working with DynamoDB