AWS Amazon DynamoDB

bogotobogo.com site search:

What is DynamoDB?

Amazon DynamoDB is a fully managed proprietary NoSQL database service. It uses synchronous replication across multiple datacenters for high durability and availability.

As a NoSQL DB, it is usually compared with Hadoop or MongoDB (DynamoDB vs. Hadoop vs. MongoDB).

Picture credit : All Things Distributed.

Features:

Managed NoSQL database.
Provisioned throughput.
Fast, predictable performance.
Fully distributed, fault tolerant.
JSON support.

Note

This tutorial is based on Getting Started with Amazon DynamoDB.

We'll work through this tutorial using the downloadable version of DynamoDB, including an interactive JavaScript shell.

This lets us learn about the DynamoDB API for free, without having to pay any fees for throughput, storage, or data transfer.

Download and Run DynamoDB

DynamoDB is available as an executable .jar file.

Download DynamoDB for free using one of these links:

$ wget http://dynamodb-local.s3-website-us-west-2.amazonaws.com/dynamodb_local_latest.tar.gz
$ gunzip dynamodb_local_latest.tar.gz 
$ tar xvf dynamodb_local_latest.tar

To start DynamoDB

$ java -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar -sharedDb
Initializing DynamoDB Local with the following configuration:
Port:	8000
InMemory:	false
DbPath:	null
SharedDb:	true
shouldDelayTransientStatuses:	false
CorsParams:	*

We can now access the built-in JavaScript shell.
URL: http://localhost:8000/shell

Set up the AWS SDK for Ruby

We can choose languages: Java, .NET, Node.js, PHP, Python, or Ruby.

In this tutorial, our choice is Ruby.

Set up the AWS SDK for Ruby:

$ sudo apt-get install ruby-full

AWS SDK for Ruby is modularized into multiple gems, each of which offers specific functionality:

$ gem install aws-sdk

'aws-sdk' is the main gem of the SDK. It contains two gems 'aws-sdk-core' and 'aws-sdk-resources', which offer two different styles of programming over AWS APIs:

$ gem install aws-sdk-core

The Core gem, 'aws-sdk-core', provides full one-to-one mapping to AWS APIs, in an RPC-style programming model. It also has a number of new built-in features such as automatic response paging, waiters, parameter validation, and Ruby type support in the Amazon DynamoDB client:

$ gem install aws-sdk-resources

The Resources gem, 'aws-sdk-resources', provides an object-oriected abstraction over the "low-level" or RPC-style interface in the Core, for a simpler and more intuitive coding experience.

A resource object is a reference to an AWS resource (such as an Amazon EC2 instance or an Amazon S3 object) that exposes the resource's attributes and actions as instance variables and methods.

Supported services include Amazon EC2, Amazon S3, Amazon SNS, Amazon SQS, AWS IAM, Amazon Glacier, AWS OpsWorks, and AWS CloudFormation, and more services will continue to be added.

Creating a Table

We'll create a table named Movies. The primary key for the table is composed of the following two attributes:

year - The partition key.
title - The sort key.

We set the endpoint (endpoint: "http://localhost:8000") to indicate that we are creating the table in DynamoDB on our computer.

In the create_table call, we specify table name, primary key attributes, and its data types.

The provisioned_throughput parameter is required; however, the downloadable version of DynamoDB ignores it. (Provisioned throughput is beyond the scope of this exercise.)

$ ruby MoviesCreateTable.rb
Created table. Status: ACTIVE

Loading Sample Data

Now we want to populate the Movies table with sample data.

We use a sample data file that contains information about a few thousand movies from the Internet Movie Database (IMDb). The movie data is in JSON format, as shown in the following example.

For each movie, there is a year, a title, and a JSON map named info.

In the JSON data, note the following:

We use the year and title as the primary key attribute values for our Movies table.
We store the rest of the info values in a single attribute called info. This program illustrates how we can store JSON in a DynamoDB attribute.

The following is an example of movie data:

{
    "year" : 2013,
    "title" : "Turn It Down, Or Else!",
    "info" : {
        "directors" : [
            "Alice Smith",
            "Bob Jones"
        ],
        "release_date" : "2013-01-18T00:00:00Z",
        "rating" : 6.2,
        "genres" : [
            "Comedy",
            "Drama"
        ],
        "image_url" : "http://ia.media-imdb.com/images/N/O9ERWAU7FS797AJ7LU8HN09AMUP908RLlo5JF90EWR7LJKQ7@@._V1_SX400_.jpg",
        "plot" : "A rock band plays their music at high volumes, annoying the neighbors.",
        "rank" : 11,
        "running_time_secs" : 5215,
        "actors" : [
            "David Matthewman",
            "Ann Thomas",
            "Jonathan G. Neff"
       ]
    }
}

Download the Sample Data File : moviedata.zip.

$ wget http://docs.aws.amazon.com/amazondynamodb/latest/gettingstartedguide/samples/moviedata.zip

After downloading the sample data, we can run the following program to populate the Movies table.

MoviesLoadData.rb:

require "aws-sdk-core"
require "json"

Aws.config.update({
  region: "us-west-2",
  endpoint: "http://localhost:8000"
})

dynamodb = Aws::DynamoDB::Client.new

tableName = 'Movies'

file = File.read('moviedata.json')
movies = JSON.parse(file)
movies.each{|movie|

    params = {
        table_name: tableName,
        item: movie
    }

    begin
        result = dynamodb.put_item(params)
        puts "Added movie: #{movie["year"]} #{movie["title"]}"

    rescue  Aws::DynamoDB::Errors::ServiceError => error
        puts "Unable to add movie:"
        puts "#{error.message}"
    end
}

Type the following command to run the program:

$ ruby MoviesLoadData.rb
...
Added movie: 2010 The Clinic
...

Query and Scan the Data

WE can use the query method to retrieve data from a table. We must specify a partition key value but the sort key is optional.

To query all movies released in a year we may want to run MoviesQuery01.rb:

require "aws-sdk-core"

Aws.config.update({
  region: "us-west-2",
  endpoint: "http://localhost:8000"
})

dynamodb = Aws::DynamoDB::Client.new

tableName = "Movies"

params = {
    table_name: tableName,
    key_condition_expression: "#yr = :yyyy",
    expression_attribute_names: {
        "#yr" => "year"
    },
    expression_attribute_values: {
        ":yyyy" => 1985 
    }
}

puts "Querying for movies from 1985.";

begin
    result = dynamodb.query(params)
    puts "Query succeeded."
    
    result.items.each{|movie|
         puts "#{movie["year"].to_i} #{movie["title"]}"
    }

rescue  Aws::DynamoDB::Errors::ServiceError => error
    puts "Unable to delete table:"
    puts "#{error.message}"
end

Run the code:

$ ruby MoviesItemQuery01.rb
Querying for movies from 1985.
Query succeeded.
1985 A Nightmare on Elm Street Part 2: Freddy's Revenge
1985 A Room with a View
...