AWS SQS NodeJS Example

bogotobogo.com site search:

What is Simple Queue Service (SQS)

We can quickly get started with SQS by using the AWS SDKs, samples, and how-tos. From Java and Ruby to .NET and Python, you can use SQS in your applications in a few lines of code.

In our case, we'll use NodeJS, and the code is available from Github (https://github.com/Einsteinish/aws-sqs-node-js-example).

Setup

Follow the instructions below on an EC2 Ubuntu instance to setup the AWS SDK using NodeJS:

sudo su
apt-get update
apt-get upgrade -y
apt-get dist-upgrade -y
apt-get autoremove -y
apt-get install nodejs npm git -y
ln -s /usr/bin/nodejs /usr/bin/node
git clone https://github.com/Einsteinish/aws-sqs-node-js-example.git
cd aws-sqs-node-js-example

Now we want to install express and aws-sdk:

npm install

Then, we may want to put our secret key info to config.json:

cp config-sample.json config.json

Running Node App

Let's start our node app:

# node app.js
AWS SQS example app listening at http://:::80

We can see it's running on port 80.

Let's look at the code in app.js:

...
// Creating a queue.
app.get('/create', function (req, res) {
    var params = {
        QueueName: "MyFirstQueue"
    };

    sqs.createQueue(params, function(err, data) {
        if(err) {
            res.send(err);
        }
        else {
            res.send(data);
        }
    });
});
...

On the browser, type in "create" with public ip address. Then, we'll get a response back:

Now we have "QueueUrl", and we can see we have "MyFirstQueue" in SQS console:

Queue Actions : SQS Message Attributes

To use Amazon SQS Message Attributes, please check Using Amazon SQS Message Attributes

QueueUrl with app.js

Let's copy the "QueueUrl" so that we can use it in app.js:

// Require objects.
var express  = require('express');
var app      = express();
var aws      = require('aws-sdk');
var queueUrl = "https://sqs.us-west-1.amazonaws.com/526262051452/MyFirstQueue";
var receipt  = "";
...

After putting in the "QueueUrl", let's restart our app:

# node app.js
AWS SQS example app listening at http://:::80

Now, we want to send a message to the queue. Let's look the code that's responsible for sending a message:

...
// Sending a message.
// NOTE: Here we need to populate the queue url you want to send to.
// That variable is indicated at the top of app.js.
app.get('/send', function (req, res) {
    var params = {
        MessageBody: 'Hello world!',
        QueueUrl: queueUrl,
        DelaySeconds: 0
    };

    sqs.sendMessage(params, function(err, data) {
        if(err) {
            res.send(err);
        }
        else {
            res.send(data);
        }
    });
});
...

Sending a message

To send a message, on the browser, we can type "send" after the endpoint:

As we can see from the screen shot, we got a valid response and a "MessageID", and our message is the message queue. We can check it from SQS:

After refreshing the page, we can notice we have 1 message is available from the "Message Available" column.

Receiving a message from the queue

Now that we know we have the message in the queue, let's try to receive the message from the queue. Again, the code snippet of app.js for the "receive":

...
// Receive a message.
// NOTE: This is a great long polling example. You would want to perform
// this action on some sort of job server so that you can process these
// records. In this example I'm just showing you how to make the call.
// It will then put the message "in flight" and I won't be able to 
// reach that message again until that visibility timeout is done.
app.get('/receive', function (req, res) {
    var params = {
        QueueUrl: queueUrl,
        VisibilityTimeout: 600 // 10 min wait time for anyone else to process.
    };

    sqs.receiveMessage(params, function(err, data) {
        if(err) {
            res.send(err);
        }
        else {
            res.send(data);
        }
    });
});
...

We can check if we received the message by using "receive" API:

We can see we got the "Hello world!" message.

On SQS console, we can see there is "1" in the "Message in Flight" column:

A message is considered to be in flight after it's received from a queue by a consumer, but not yet deleted from the queue.

For standard queues, there can be a maximum of 120,000 inflight messages per queue. If we reach this limit, Amazon SQS returns the OverLimit error message. To avoid reaching the limit, we should delete messages from the queue after they're processed. We can also increase the number of queues we use to process our messages.

Deleteing a message - working with ReceiptHandle

Let's copy the ReceiptHandle:

Then, put it into app.js like this:

// Require objects.
var express  = require('express');
var app      = express();
var aws      = require('aws-sdk');
var queueUrl = "https://sqs.us-west-1.amazonaws.com/526262051452/MyFirstQueue";
var receipt  = "AQEBOQmPpQ3BI7lry0Dyx3kyyVoPrCklOEc7j5OrQK3btzR9IJJNz5wEb3el85bU36yXSVzWhedaY9y8IcVd3QOq8SUpnm4pQPTJx9g62pulH7OaG1kJboIrvHWMG9Sh4+Vk2+lcS9hus+WiO5cqdVY1Ed642Tnnxzg2GUEGKi62mBkBf7g3o1SuXei6FNF+Ld9IjoAhX6hYTnI/2l0IX//XI2p1ukce9PHriTf5DuIVNSCwY1jx6dr7YefCD8gXimYmSR9BPI6jmA6Vfs3+074z9q1U+zWQbcJnxxXbTnzLQQ4mPHHpqi2s2emGl+1WoddUAV27I3LhoyjNlmb4URI9GEirOifr+jfZ+wbSymjk/BzZKDvO9greJpmiF62EvpuPrPnvpAUrdGt2FXmxFVToog==";
...

Here is the delete portion of the code in app.js:

...
// Deleting a message.
app.get('/delete', function (req, res) {
    var params = {
        QueueUrl: queueUrl,
        ReceiptHandle: receipt
    };

    sqs.deleteMessage(params, function(err, data) {
        if(err) {
            res.send(err);
        }
        else {
            res.send(data);
        }
    });
});
...

Make sure we restart the app, then we can try to delete the message: