Amazon DynamoDB Lessen's Learned by Beginner

75 %
25 %
Information about Amazon DynamoDB Lessen's Learned by Beginner

Published on March 18, 2014

Author: HirokazuTokuno



Presented 14-Mar-2014 at Sendai, JAWS-UG.

Amazon DynamoDB Lessen's Learned by Beginner Correct me if anything wrong JAWS-UG Aizu Tokuno

About me ● TOKUNO, Hirokazu ● IT department in factory ● 4th years moved to IT at age of 39 – Studying hard to catch up ● Little know about – VBA, SQL, JavaScript, Java, Perl, Shell... ● Like to not write code – Pentaho ETL, Oracle APEX, Mojolicious

Learning/Challenge in 2013 ● Unicage Development Method ● Perl, Mojolicious ● Jenkins ● Automate Test jUnit, Mockito, shUnit2 ● Twitter Bootstrap, Google Maps API etc. ● Fun to attend workshop/seminar! – efsta! – Open Data Cafe Aizu – JAWS-US Aizu ● Join to CODE for AIZU

Learning/Challenge in 2014 ● Enroll to Linked Open Data Challenge 2013 ● Attend Hack for Town in Aizu ● Meteor(Node.js, MongoDB) on Nitrous.IO ● GitHub Debut ● Amazon DynamoDB Right now!

No experience in Could

Created AWS account just 7 days ago

What's DynamoDB? ● Fully Managed NoSQL Database Service

What's DynamoDB? ● Fully Managed NoSQL Database Service What's the hell are you talking about?

What's DynamoDB? ● Fully Managed NoSQL Database Service – NoSQL – Fully Managed Database Service Split into peaces

Understanding of NoSQL ● Not only SQL ● Fast to read from huge amount of data – Accept inconsistent ● Flexible Data Structure (DynamoDB etc.) – Attributes can be differ for each record ● Not suitable for summary and query

Fully Managed DB Service ● Rent database service as server – Pay for read/write speed, storage, transaction ● Managed by Amazon – Install, Upgrade and Patching – Keep storage space, no limit! – Distribute 3 data centers – Improve performance

Sum up! ● Ask Amazon for boring tasks! ● Extreme performance with SSD! ● Money solve the issue! ● Not to think about storage limitation! ● High reliability by distributing at 3 data center! ● Be careful! It's not traditional database.

Let's try! ● Service - DynamoDB

Create Table ● Create Table

Configure Table ● Primary Key – Only Hash or Hash/Range combination

Primary Key ● Hash Key : Column for partitioning – Pick not converged column like UserID – Do not use picky column like Game ID ● Range Key : – Column for search ● Push into 2 columns to be unique

Example of Primary Keys ● Present forum in 3 tables Table Primary Key Attributes Hash Range Group JAWS-UG { LastPostBy="Kurihara" LastPostDateTime= "2012-01- 03T00:40:57.165Z" } Thread JAWS-UG Session { Message = "How about unconference?" Tags = [ "JAWS-UG", "Aizu" ] LastPostDateTime = "2012-01- 03T00:40:57.165Z" } Reply JAWS- UG#Session 2011-12- 11T00:40:57. 165Z { Message = "Extreeeeeeme!" PostedBy = "Kurihara" }

Attributes ● Contents other than Hash, Range ● Each entry may have own attributes ● Data Type – String – Number – Binary – String Set (Array) – Number Set (Array) – Binary Set (Array) Convert Date to string

Create Index ● May skip since optional

Index ● Upto 5 indexes can be attached for each table – Hash-Range table is created under the table – Cost for read/write and storage ● Able to create at table creation. No edit, add. ● Local secondary index – Create Hash-Range table with shared Hash Key as table, Range Key is not ● Global secondary index – Create Hash-Range table with non shared Hash and Range

Index ● Sounds working like this... Hash Range Attribute1 Attribute2 A 1 abc def B 2 ghi jkl Hash Attribute2 Attribute1 Range A def abc 1 B jkl ghi 2 Primary Key Primary Key Index

Throughput ● Read Capacity Unit – Able to read one item per second for 4KB ● Write Capacity Unit – Able to write one item per second for 1KB

Alarm ● Send notification if exceed specified threathold, 75~95%

Final confirmation ● As usual ¥60/month!

Complete! ● 3 tables for forum

Disadvantages ● Unable to modify Index – Need to re-build if need to modify – Requires cost to migrate data ● (Possibly) No system to sync with doc like, ERD ● Suitable application and not suitable – Need to scan all records to query by othet than Primary Key, Index – Requires RDB for consistent transaction or use Java Transaction Library – Complex with RDB as recommended

Access from programs! ● Official SDKs ● Unofficial SDK for Perl etc. Android Browser iOS Java .NET Node.js Python PHP Ruby

SDK API ● Table – CreateTable – DeleteTable – DescribeTable – ListTables – UpdateTable ● Batch processing – BatchGetItem – BatchWriteItem ● Item – PutItem – GetItem – UpdateItem – DeleteItem ● Search – Query : Primary or Index – Scan : all record

Let's play with DynamoDB!

NoSQL is for Big Data

Big Data for Twitter

Make clear what's interesting!

What is to be investigated...

Voluntary Restraints!!

Okay, okay, let's try...

Forum with WebSock

Too many samples available... Not challenging enough

Create RESTfull API!

Like this DynamoDBNitrous.IOWeb Client ● All free of charge

RESTfull API ● Implemented very simple function – No BatchGetItem/BatchWriteItem Action Function Method Example of URL Create putItem POST /Group/Hash?p=1&q=A Read scan getItem GET /Groups /Groups/Hash Update updateItem PUT /Group/Hash?p=A&q=1 Delete deleteItem DELETE /Group/Hash

Create by POST ● curl -i -X POST -H "Content-Type: application/json" -d '{"datetime":"2013-03- 13T23:27:32.256Z" , "by":"Tokuno"}' http://your_host:8080/Group/JAWS-UG

Read all by GET ● curl -H "Content-Type: application/json" http://your_host:8080/Groups

Read by GET ● curl -H "Content-Type: application/json" http://your_host:8080/Group/JAWS-UG May need URI encode for Japanese

Update by PUT ● curl -i -X PUT -H "Content-Type: application/json" -d '{"datetime":"2021-03- 14T00:04:01.951Z" , "by":"No one tell me..."}' http://your_host:8080/Group/JAWS-UG

Delete by DELETE ● curl -i -X DELETE http://your_host:8080/Group/JAWS-UG

One more thing

Like this DynamoDBNitrous.IOPentaho ETL ● All free!

Pentaho for data processing ● Limited by DynamoDB throughput ● Pay more to boost Group LastPostDateTime LastPostBy あ 2013-03-13T23:27:32.256Z とくのう い 2013-03-13T23:27:32.256Z とくのう う 2013-03-13T23:27:32.256Z とくのう え 2013-03-13T23:27:32.256Z とくのう お 2013-03-13T23:27:32.256Z とくのう か 2013-03-13T23:27:32.256Z とくのう き 2013-03-13T23:27:32.256Z とくのう く 2013-03-13T23:27:32.256Z とくのう け 2013-03-13T23:27:32.256Z とくのう こ 2013-03-13T23:27:32.256Z とくのう あ1 2013-03-13T23:27:32.256Z とくのう い1 2013-03-13T23:27:32.256Z とくのう う1 2013-03-13T23:27:32.256Z とくのう え1 2013-03-13T23:27:32.256Z とくのう お1 2013-03-13T23:27:32.256Z とくのう か1 2013-03-13T23:27:32.256Z とくのう き1 2013-03-13T23:27:32.256Z とくのう

Summary ● Easy to maintaing since managed by Amazon! ● NoSQL is not 'Relational' but application links – Not easy to develop ● Unable to add/modify Index ● Not flexible to query. Need to query every 1MB ● Want method to actual and document ● Only suitable for disposable application and enterprise whom has big data?

Thank you!

Timeline ● 2014.03.08 Create account ● 2014.03.11 Attend Hack for Town 2014.03.11 Cross over in last 3 years ● 2014.03.12 Presentation for Game Lab ● 2014.03.13 Develop RESTful 2014.03.13 Play with Pentaho ● 2014.03.14 Presentation, today


Region ● Find region used

Procedure to find Credentialss


Create a New Group of Users


Permissions ● Select Power User Access for full access

Verify Policy ● Required to review although nonsense

Create User


Credentials ● Finally

Add a comment

Related presentations

Presentación que realice en el Evento Nacional de Gobierno Abierto, realizado los ...

In this presentation we will describe our experience developing with a highly dyna...

Presentation to the LITA Forum 7th November 2014 Albuquerque, NM

Un recorrido por los cambios que nos generará el wearabletech en el futuro

Um paralelo entre as novidades & mercado em Wearable Computing e Tecnologias Assis...

Microsoft finally joins the smartwatch and fitness tracker game by introducing the...

Related pages

Lessons to Be Learned: Musik Gabriella Cilmi – Lessons to Be Learned jetzt kaufen. ... Lessons to be learned ist ein guter Start für Gabriella Cilmi, ...
Read more

Amazon DynamoDB – a Fast and Scalable NoSQL Database ...

Amazon DynamoDB stores data on Solid State Drives (SSDs) and replicates it synchronously across multiple AWS ... Lessons learned from Amazon's Dynamo.
Read more Customer Reviews: Composite Construction for ...

... Construction for Beginners: A Practical Application of Lessons Learned Studying and Working with High Performance Composites at Read ...
Read more

Amazon's Dynamo - All Things Distributed

Experiences & Lessons Learned. Dynamo is used by several services with different configurations. ... Amazon DynamoDB ...
Read more

Composite Construction for Beginners: A Practical ...

Composite Construction for Beginners: A Practical Application of Lessons Learned ... A Practical Application of Lessons Learned Studying ... Amazon Best ...
Read more

What are the lessons learned from Amazon's June 14th, 2012 ...

What are the lessons learned from Amazon's June 29th us-east-1 outage? What are the lessons learned from April 21, ... Amazon DynamoDB: ...
Read more

Amazon Web Services (AWS) - Cloud Computing Services

Amazon DynamoDB . 25GB of storage and up to 20 million requests/month. ... Amazon Web Services is Hiring. Amazon Web Services (AWS) is a dynamic, ...
Read more

AWS re:Invent 2015 | Sessions

... and Solutions Architects or expert customers who can share their real world experiences and lessons learned. ... Amazon DynamoDB, Amazon Redshift ...
Read more