Introduction

This article entails what a lambda function and what cloud formation is and their use cases. We will then deploy a lambda function that will be triggered when an object is uploaded into an S3 bucket and sends out an SNS email notification upon success.

What is AWS Lambda?

AWS Lambda is an event-driven, serverless computing platform provided by AWS. It reduces the overhead of having to provision and configure the underlying infrastructure.

On the other hand, a lambda function is a script or program that is run in AWS Lambda. Events that are invoked are sent to the program from Lambda and it returns a response.

AWS Lambda supports various programming languages like Python, NodeJs, Ruby, Go, Java and .NET, which are used as the functions runtime.

AWS Lambda Use Cases

According to AWS, Lambda has quite a number of use cases like:

• Processing uploaded S3 objects, which I'm going to implement in this article,
• Serverless websites with AWS Lambda, API Gateway and DynamoDB,
• Real-time notifications with AWS Lambda and SNS,
• Real-time data analysis with AWS Lambda and Kinesis Data Streams, and so on.

What is Cloud Formation

Cloud Formation is an Infrastructure as Code tool provided by AWS. It is a simple way used to provision, update, and destroy a collection of AWS Infrastructure consistently.

AWS Cloudformation allows you to manage your complete AWS infrastructure in a template file written in YAML or JSON format

Overview of the Lambda Function

In this article, I'll deploy a function that processes objects that are uploaded to an s3 bucket and sends an SNS notification upon success. Here is a pictorial representation of what I'll be doing.

Implementing this function using cloudformation can be quite tricky due to very frequent circular dependency errors which is why I'll be deploying it step by step and not all at once.

Prerequisites

• Make sure AWS CLI is installed, if not you can get it here.

• Configure AWS CLI with a user that has the necessary roles attached or an admin role. You can learn how to configure AWS cli here.

• Set the region to 'us-east-1'.

Steps

• Create an IAM role with policies that grants lambda access to the s3 bucket and SNS,

• Create the lambda function,

• Create the SNS Topic and subscription,

• Create the S3 bucket and a lambda permission that allows the s3 bucket to trigger the lambda function.

Creating the Lambda IAM role and Lambda Function

Below you'll find a github gist with a template file, copy and paste it into a .yml file

Also, copy this script into a parameter.json file. This is the parameters file that is referenced in the Parameters block of the template file. Make sure to replace with your email address.

[
    {
        "ParameterKey": "SNSEndpoint",
        "ParameterValue": "<email-address>"
    }
]

Let's break this script down now shall we:

The first block in the template file is the Parameters block. Parameters enable you to input custom values to your template each time you create or update a stack.

The next block of the script is the Resources block. This includes the AWS resources we want to create. The first resource is the LambdaRole. This IAM role has the necessary policies attached to it that allows the lambda function creates logs, get the objects uploaded to an S3 bucket and publish messages to an SNS topic and its subscribers. The second resource is the LambdaFunction itself which has the IAM role attached to it. The function gets the object from the event and shows its content type. The next two resources, LambdaSNS and SnsSubscription, creates an SNS topic and SNS subscription with an email address as the endpoint. The email address will be in the parameter.json file. The last resource which is the LambdaInvokeSNS is where we configure our asynchronous invocation, i.e upon success of the event, lambda invokes the SNS topic which then sends us an email.

The Output part of the script is where we export the value of our function Arn so that we can cross reference when creating the S3 bucket and its permissions.

Save the file and then run the command below to create the resources. Replace the with a preferred name for the cloud formation stack and with the name of the template file.

aws cloudformation create-stack --stack-name <stack-name> --template-body file://<file-name>.yml  --parameters file://parameter.json --capabilities CAPABILITY_IAM

You should get an output showing the stack-id.

Go to your AWS account, then cloudformation. You should see that a stack has been created and in a few minutes, It shows 'Create Complete'.

You can also switch to the resources tab of the stack to confirm that all resources have been created.

Check the inbox of the email address you used to subscribe to the SNS topic and make sure to click on 'Confirm Subscription'

Creating the S3 bucket and its lambda permission

This part is a little bit tricky. We have to create the S3 bucket first... Copy and paste this into a .yml file

Parameters:
  BucketName:
    Description: Unique name for s3 bucket
    Type: String
Resources:
  LambdaBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Ref BucketName
      PublicAccessBlockConfiguration:
        RestrictPublicBuckets: true
        BlockPublicAcls: true
        IgnorePublicAcls: true
        BlockPublicPolicy: true

Copy and paste this into another .json file. This is the parameters file for the S3 bucket. Make sure to replace with a unique name for the bucket.

[
    {
        "ParameterKey": "BucketName",
        "ParameterValue": "<bucket-name>"
    }
]

Save the file and run the command below to create the S3 bucket

aws cloudformation create-stack --stack-name <stack-name> --template-body file://<file-name> --parameters file://<parameters-file>

Go back and check the cloudformation stacks and you should see that another stack has been created and its completed

Then we create the LambdaPermissions. The AWS::Lambda::Permission resource grants the S3 bucket permission to use a function.

Add the script below to the s3 template file as another resource

LambdaBucketPermission:
    Type: AWS::Lambda::Permission
    Properties:
      Action: lambda:InvokeFunction
      FunctionName: !ImportValue LambdaFunctionArn
      Principal: s3.amazonaws.com
      SourceArn: 
        Fn::GetAtt:
          - LambdaBucket
          - Arn

Run the command below to update the cloudformation stack

aws cloudformation update-stack --stack-name bucket --template-body file://s3.yml --parameters file://s3-param.json

The cloud formation stack would be updated and in a short while show 'Update Complete'.

Now, we'll go back and update the bucket resource by adding a lambda notification configuration to the bucket. The LambdaConfiguration describes the AWS Lambda functions to invoke and the events for which to invoke them. The updated block would look like this,

Resources:
  LambdaBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Ref BucketName
      PublicAccessBlockConfiguration:
        RestrictPublicBuckets: true
        BlockPublicAcls: true
        IgnorePublicAcls: true
        BlockPublicPolicy: true
      NotificationConfiguration:
        LambdaConfigurations:
          - Event: s3:ObjectCreated:*
            Function: !ImportValue LambdaFunctionArn

Update the cloudformation stack and confirm from the AWS console that the update was completed.

Putting it all together and testing the function

On the AWS console, go to S3, click on the bucket that was just created. We will upload a sample file into the bucket and see our function in action. You can upload any kind of file, be it a .txt or .sh or any simple file.

Go to Lambda, click on the function we just created, scroll down and go to the monitor tab. If you don't see any metrics yet, wait for a while and refresh.

You should see that the function was invoked and it was successful. Also check the inbox of the email address you used earlier, you should get a mail that looks like this

Conclusion

You have just learnt how to deploy a lambda function, an s3 bucket and SNS topic using cloud formation. The github link to all the files can be found here. Make sure you delete all stacks to avoid any extra charges.