{"componentChunkName":"component---src-templates-blog-js","path":"/ctf/aws-formation/tp8-dynamodb","result":{"data":{"markdownRemark":{"html":"

TP8 - DynamoDB

\n

1. Introduction to DynamoDB

\n

Definition

\n

DynamoDB = fast flexible NoSQL database service

\n

Services with :

\n\n

Supports 2 data models:

\n
    \n
  1. document
    2. \n
  2. key-value
    3. \n
\n

It is serverless, integrate with lambda\nIt avoids single point of failure (spread accross 3 distinct data centers)

\n

2 read consistency models :

\n
    \n
  1. Eventual (defaults) : best read performance, actualisation within a second
    2. \n
  2. Strongly
    3. \n
\n

Tables

\n\n

Documents can be writtent in JSON, HTML or XML

\n\n

Primary keys

\n

DynamoDB stores and retieves data based on a primary key

\n

2 types of primary keys :

\n
    \n
  1. \n

    partition keys = unique attributes

    \n\n
    2. \n
  2. \n

    composite key = partition keys (userID) + sort key (timestamp of the post)

    \n\n
    3. \n
\n

\"TP8-DynamoDB-image-20200722202256124\"

\n

Access control

\n\n

You can add a condition to an IAM Policy to allow access only to items where the partition key value matches their UserID

\n

\"TP8-DynamoDB-image-20200722202833266\"

\n

Partiton key = Leading Key

\n

remember

\n\n

2. Creating a DynamoDB Table Lab

\n

Steps

\n
    \n
  1. Create IAM Service Role for EC2 for DynamoDB Full Access
    2. \n
  2. Create EC2 (by configuring instance initializing a php website)
    3. \n
  3. Connect to EC2 by ssh
    4. \n
  4. Install [AWS SDK for PHP version 2] (https://docs.aws.amazon.com/aws-sdk-php/v2/guide/installation.html) by downloading the composer
    5. \n
  5. Change the region of uploaddate.php
    6. \n
  6. Create the DynamoDB with IPaddress/dynamoDB/createTables.php
    7. \n
\n

how to interact with the database using the command line ?

\n\n

3. Indexes Deepdive

\n

index definition

\n\n

\"TP8-DynamoDB-image-20200723011044767\"

\n

4. Scan vs Query API Call

\n

A query and a scan return all the attibutes of the items but you can use projection expression to select a specific attributes.

\n

Query

\n

A query finds items in a table based on the primary key attributes and a distinct value to search for.

\n\n

By default queries are eventually consistent

\n

Scan

\n

A scan operation examines every item in the table\nBy default returns all the attributes

\n

Comparison

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
QueryScan
Efficiency+-
Dump the entire table ?noyes
Can use up the provisioned throughput ?noyes
\n

improve performance

\n\n

of scan

\n

share your datable into segments and scan them in parallel

\n

5. DynamoDB Provisioned Throughput

\n

DynamoDB Provisioned Throughput is measured in Capacity Units

\n

When you create a tabl, you specify your requirements in terms of Read Capacity Units and Write Capacity Units.

\n\n

steps to know how many do we need

\n
    \n
  1. calculate how many Read CU needed for each read : Size of each item / 4 Kb
    2. \n
  2. Rounded-up
    3. \n
  3. Multiply by the number of read per second
    4. \n
\n

6. DynamoDB On Demand Capacity

\n\n

pricing model shoul I use ?

\n

\"TP8-DynamoDB-image-20200723013721046\"

\n

you can change of pricing model once a day.

\n

7. DynamoDB Accelerator (DAX)

\n

definition

\n

*DAX = is a fully managed clustered in-memory cached for DynamoDB

\n

Delivers up to a 10x read performance improvments = microsecond performance for millions of request per sec (Christmass or Black Friday) -> Ideal for Read-Heavy and bursty workloads

\n

how it works

\n

If the item is not available (cache miss) then DAX performs an Eventually Consistent GetItem operation against DynamoDB

\n

NOT suitable for

\n\n

9. DynamoDB Transactions

\n

ACID transactions : Atomic (single unit, all or nothing operations) Consistent (must let datatable in a valid state) Isolated (not dependency in transaction) Durable (when a transaction have been commited it will remain in the data table)

\n

Read or write multiple items across multiple tables as an all or nothing operations.

\n

10. DynamoDB TTL

\n

definition

\n

TTL : Time To Live defines an expiry time for your data

\n

It is relevant to remove old data (session data event logs ...) and so reduce cost storage data

\n

It is expressed as POSIX (Unix Time = )\nEpochConverter\nL'heure Unix ou heure Posix (aussi appelée Unix timestamp) est une mesure du temps basée sur le nombre de secondes écoulées depuis le 1er janvier 1970 00:00:00 UTC, hors secondes intercalaires.\nLes quatre premières lettres forment l’acronyme de Portable Operating System Interface (interface portable de système d'exploitation), et le X exprime l'héritage UNIX.

\n

select items on TTL

\n

steps :

\n
    \n
  1. check your IAM user permissions (aws iam get-user)
    2. \n
  2. create a sessionData table (aws dynamodb create-table)
    3. \n
  3. populate sessionData table (aws dynamodb batch-write-item)
    4. \n
\n

11. DynamoDB Streams

\n

definition

\n

It s a time ordered sequece (or streams).\nSo any modification at the item level (insert update, delete) will be saved in the DDB streams with a encrypted log during 24h.\nThey are used to trigger event or lambda event based on a change of the DDB table\nBy default the primary key is recorded.\nBefore and After images can be captured.

\n

\"TP8-DynamoDB-image-20200723020935605\"

\n

The DDB endpoint is different than DDB streams endpoint

\n

12. Provisioned Throughput Exceeded & Exponential Backoff

\n

If you see Provisioned Throughput Exceeded error it means the number of request is too high

\n

Provisioned Throughput Exceeded exception

\n

If you request rate is too high for the read / write capacity provisionned on you DDB table. SDK will automatically retries requests untill successful

\n

exponential backoff

\n

If we do not use SDK, we can :

\n\n

Exponential backoffs improves flow by retrying request using progressively longer waits (10 20 40 80 ms...)\nIf the waiting exceed 1min, your request may exced the throughtput of your red/Write capacity.

","frontmatter":{"date":"July 31, 2020","path":"ctf/aws-formation/tp8-dynamodb","title":"TP8 - DynamoDB","tags":["cloud","ec2","aws"],"categorie":"ctf","thumbnail":"/assets/alex-machado-80sv993luki-unsplash.jpg"},"fields":{"readingTime":{"text":"8 min read"}}},"file":{"childImageSharp":{"fluid":{"base64":"data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAANABQDASIAAhEBAxEB/8QAFwAAAwEAAAAAAAAAAAAAAAAAAAMEBf/EABUBAQEAAAAAAAAAAAAAAAAAAAED/9oADAMBAAIQAxAAAAGFz11jKaIn/8QAGhAAAgMBAQAAAAAAAAAAAAAAAAECERMSMf/aAAgBAQABBQLm3gxwaJTxe1i8/8QAGBEAAgMAAAAAAAAAAAAAAAAAAAECElH/2gAIAQMBAT8BUdKn/8QAFhEBAQEAAAAAAAAAAAAAAAAAABEB/9oACAECAQE/Abqv/8QAGhAAAQUBAAAAAAAAAAAAAAAAAQACEBESIv/aAAgBAQAGPwJckOjObVBtR//EABwQAQACAgMBAAAAAAAAAAAAAAEAETFBIWFxkf/aAAgBAQABPyEVB9ih2jUSxfkFQDkWX4Ju5grU/9oADAMBAAIAAwAAABDjD//EABYRAQEBAAAAAAAAAAAAAAAAAAEAIf/aAAgBAwEBPxAA1M//xAAWEQEBAQAAAAAAAAAAAAAAAAABIRD/2gAIAQIBAT8QWYTP/8QAGhABAQEAAwEAAAAAAAAAAAAAAREAIUFxwf/aAAgBAQABPxAYkFlcFzRnCFCH3zCCggiKI5gk6hL1lpwEhs0zEABv/9k=","aspectRatio":1.4970059880239521,"src":"/static/11cdcb302d032fa2db3a46428a8f74c9/a7715/alex-machado-80sv993luki-unsplash.jpg","srcSet":"/static/11cdcb302d032fa2db3a46428a8f74c9/8f7df/alex-machado-80sv993luki-unsplash.jpg 250w,\n/static/11cdcb302d032fa2db3a46428a8f74c9/0f3a1/alex-machado-80sv993luki-unsplash.jpg 500w,\n/static/11cdcb302d032fa2db3a46428a8f74c9/a7715/alex-machado-80sv993luki-unsplash.jpg 1000w,\n/static/11cdcb302d032fa2db3a46428a8f74c9/37d86/alex-machado-80sv993luki-unsplash.jpg 1500w,\n/static/11cdcb302d032fa2db3a46428a8f74c9/a41d1/alex-machado-80sv993luki-unsplash.jpg 2000w,\n/static/11cdcb302d032fa2db3a46428a8f74c9/6a059/alex-machado-80sv993luki-unsplash.jpg 6000w","sizes":"(max-width: 1000px) 100vw, 1000px"}}}},"pageContext":{"slug":"ctf/aws-formation/tp8-dynamodb","featuredImage":"alex-machado-80sv993luki-unsplash.jpg"}}}