Build a Serverless App using SNS, SQS and Lambda on AWS

🤖 Introduction

In this AWS tutorial, you'll learn how to build a simple event-driven and serverless application using Amazon SNS, Amazon SQS, and AWS Lambda.

📐 Architecture Design

⚙️ AWS Services Used

• Amazon Simple Notification Service (Amazon SNS)
• Amazon Simple Queue Service (Amazon SQS)
• AWS Lambda Function
• Amazon CloudWatch

🔋 Features

👉 The user is going to submit a notification to an SNS topic

👉 It's going to be integrated with a queue in other words the queue is subscribed to the topic and the message that we add to the topic ends up in the queue

👉 Then SQS is going to trigger a Lambda function

👉 The Lambda function is going to run and it's going to write some information to cloudwatch and whatever we put in the topic we're going to see that in Cloud watch logs

➡️ Step 1 - Create a queue

We're going to use Amazon SQS by creating a queue, sending a message to the queue, and receiving and processing the message.

To create a queue:

1. Navigate to the Amazon SQS and click on "Create queue"

2. Choose "Standard" as a queue type, name the queue MyQueue

3. I'm not going to change any of parameters we're just going to leave the configuration as default, then scroll down and click "Create queue"

➡️ Step 2 - Create a topic

A topic is a message channel. When you publish a message to a topic, it fans out the message to all subscribed endpoints.

To create a topic:

1. Navigate to the Amazon SNS console, you can go over to topics on the left hand side and click "Create topic"

2. Choose "Standard" and I'll simply name demo-sns then sroll down, don't need to change anything else and click "Create topic"

3. There's now an option to create a subscription so let's click on " Create subscription"

4. The topic ARN is selected for us then I'm going to choose "Amazon SQS" for the protocol and select my queue as the endpoint and then click on "Create subscription"

5. The SNS does need permissions to the queue so what we can do is copy the ARN of our topic:

• let's go to visual studio code and open the access policy, what I need to do is paste it into my source ARN for my topic

• I need to go back and get the Queue ARN, let's simply copy the Queue ARN (not the URL make sure it's the ARN) and back in the visual studio and paste it next to resource.

    {
      "Statement": [
      {
        "Effect": "Allow",
        "Principal": {
            "Service": "sns.amazonaws.com"
        },
        "Action": "sqs:SendMessage",
        "Resource": "YOUR-QUEUE-ARN",
        "Condition": {
            "ArnEquals": {
            "aws:SourceArn": "YOUR-ARN-TOPIC"
        }
      }
    }
  ]
}

6. Copy the code and we're going to put it in as an access policy on SQS, go back to SQS console, click on "Access Policy",

7. Then click on "Edit", you will see an example of a resource based policy, now I'm going to delete out everything that's already in there and paste in my code and then click on "Save".

👍 Now the SNS does have permissions to the SQS Queue.

➡️ Step 3 - Create a Lambda function

To create a Lambda function:

1. Navigate to the Lambda console, click on "Create fonction"

2. Choose "Author from scratch", Enter a name for the function MyTest, choose runtime Node.js 16.x then click "Save"

3. Next we need to update our function code, we already have a function let's come into the function and in source code let's go to index.js and again we've got some code in visual studio so I've opened this file in the AWS Lambda directory SQS to Lambda copy all of the code to your clipboards and let's go and paste it in we'll simply paste it into the code editor and then deploy.

    exports.handler = async function(event, context) {
  event.Records.forEach(record => {
    const { body } = record;
    console.log(body);
  });
  return {};
}

👍 This Lambda function code is going to pass the messages from sqs and it's going to write whatever it finds in the body of the message into Cloud watch logs.

4. We do need to have some permissions to the queue for Lambda as well remember the function execution role is what determines the permissions that Lambda has when it executes so it must have permissions to read messages from the queue and then delete them from the queue.

• Back in Lambda let's go to configuration and then choose the execution role

• Then I'm going to click on "ADD permissions" and attach policies let's just search for SQS the select AWSLambdaSQSQueueExecutionRole and what we want is the SQS execution role this does have the receive message and delete message permissions so we'll attach that policy to the role and click on "Add permissions"

5. Lastly we're going to create the trigger in SQS:

• Go to Lambda triggers, click on "Configure Lambda function trigger"

• Choose the Lambda function and then click on Save

👍 So what have we done, we've created an SNS topic and we've subscribed an SQS Queue to the topic and we made sure that the queue has permissions to allow the topic to add messages to the queue then we updated the function code and we added permissions to receive messages and delete them from the queue and of course it already has permissions to cloudwatch so when we add a message to the topic and we'll do that manually it will then push that message to the queue Lambda will then process it and write the event to cloudwatch.

➡️ Step 4 - Let's Test our Topic

1. Back on our topic click on "Publish message"

2. We're going to enter a subject I just wrote serverless app test and then I'm going to write VOILA, IT WORKED!! and click on "Publish message"

3. What we're going to do is go to Lambda and then go back to monitor click on view logs in cloudwatch logs

• We can see a very recent execution happened I know that this is the most recent one

• If i expand each logs and we can see the information, we can see the subject and we can see the message which I added manually.

👍 That's it, a really simple serverless application that is event driven.

💰 Cost

All services used are eligible for the AWS Free Tier. However, charges will incur at some point so it's recommended that you shut down resources after completing this tutorial.