AWS Amazon DynamoDB with Lambda and CloudWatch

bogotobogo.com site search:

Introduction

Amazon DynamoDB is a fully managed proprietary NoSQL database service. It uses synchronous replication across multiple datacenters for high durability and availability.

In this post, to put data into a DynamoDB table, we'll use Write Data to a Table Using the Console or AWS CLI.

Picture credit : How to Create an AWS IAM Policy to Grant AWS Lambda Access to an Amazon DynamoDB Table

In this post, we'll do the following:

Create a DynamoDB table, Music.
Create an IAM role, DynamoDBStreamLambdaRole that has AWS policy, AWSLambdaDynamoDBExecutionRole.
Create a Lambda function, MusicEventProcessor, with the permission specified in AWSLambdaDynamoDBExecutionRole.
Set event trigger on DynamoDB. Set the trigger Lambda function as MusicEventProcessor.
Then we create some events by INSERT/MODIFY/DELETE on DynamoDB Table either via CLI or console.
Check if events have been triggered using Monitoring section of Lambda and CloudWatch Log.

Create a DynamoDB table

Create a Music table in Amazon DynamoDB with the following details:

Partition key — Artist
Sort key — SongTitle

The rest of the steps are from Step 1: Create a Table

Create an IAM role

Create an IAM role, DynamoDBStreamLambdaRole that has AWS managed policy, AWSLambdaDynamoDBExecutionRole as shown below:

Create a Lambda function

Create a Lambda function, MusicEventProcessor, with the permission specified in AWSLambdaDynamoDBExecutionRole:

The function code looks like this:

import json

print('Loading function')

def lambda_handler(event, context):
	print('------------------------')
	print(event)
	#1. Iterate over each record
	try:
		for record in event['Records']:
			#2. Handle event by type
			if record['eventName'] == 'INSERT':
				handle_insert(record)
			elif record['eventName'] == 'MODIFY':
				handle_modify(record)
			elif record['eventName'] == 'REMOVE':
				handle_remove(record)
		print('------------------------')
		return "Success!"
	except Exception as e: 
		print(e)
		print('------------------------')
		return "Error"


def handle_insert(record):
	print("Handling INSERT Event")
	
	#3a. Get newImage content
	newImage = record['dynamodb']['NewImage']
	
	#3b. Parse values
	newArtist = newImage['Artist']['S']
	newSongTitle = newImage['SongTitle']['S']

	#3c. Print it
	print ('New row added with newArtistId=' + newArtist + "SongTitle=" + SongTitle)

	print("Done handling INSERT Event")

def handle_modify(record):
	print("Handling MODIFY Event")

	#3a. Parse oldImage and score
	oldImage = record['dynamodb']['OldImage']
	oldAwards = oldImage['awards']['N']
	
	#3b. Parse oldImage and score
	newImage = record['dynamodb']['NewImage']
	newAwards = newImage['awards']['N']

	#3c. Check for change
	if oldAwards != newAwards:
		print('Awards changed - oldAwards=' + str(oldAwards) + ', newAwards=' + str(newAwards))

	print("Done handling MODIFY Event")

def handle_remove(record):
	print("Handling REMOVE Event")

	#3a. Parse oldImage
	oldImage = record['dynamodb']['OldImage']
	
	#3b. Parse values
	oldArtist = oldImage['Artist']['S']

	#3c. Print it
	print ('Row removed with Artist=' + oldArtist)

	print("Done handling REMOVE Event")

Actually, I borrowed the code from AWS DynamoDB Streams to Lambda Tutorial in Python | Step by Step Guide and modifed a bit for the Music table.

Also, the OldImage or NewImage are DynamoDB specific, and we can find more info from StreamRecord.

Set event trigger on DynamoDB

Let's set event trigger on DynamoDB. Set the trigger Lambda function as MusicEventProcessor

Create some events by INSERT/MODIFY/DELETE on DynamoDB TableB

We can create some events of INSERT/MODIFY/DELETE.

We can either work on console or use CLI (Step 2: Write Data to a Table Using the Console or AWS CLI), for example, put:

aws dynamodb put-item \
--table-name Music  \
--item \
    '{"Artist": {"S": "No One You Know"}, "SongTitle": {"S": "Call Me Today"}, "AlbumTitle": {"S": "Somewhat Famous"}, "Awards": {"N": "1"}}'

aws dynamodb put-item \
    --table-name Music \
    --item \
    '{"Artist": {"S": "Acme Band"}, "SongTitle": {"S": "Happy Day"}, "AlbumTitle": {"S": "Songs About Life"}, "Awards": {"N": "10"} }'

Check if events have been triggered

Finally, we need to check if events have been triggered using Monitoring section of Lambda and CloudWatch Log.