Using Using Lambda-backed custom resources to extend the functionality of CloudFormation

This is part 3 of a 3 part series that I wrote on using CloudFormation to rollout a personal VPN internet proxy. This part guides you through the details of extending the functionality of CloudFormation using a Lambda-backed custom resource to clean the secrets bucket on stack deletion. You can find the article posted here on Cloud Assessments:

How to roll your own VPN with AWS CloudFormation – Part three

Corresponding video walkthrough that you can find here on YouTube:

The article will guide you through the content of this template:

Let’s build a serverless web app on AWS


It’s possible to build a web service using an application platform that is fully managed by AWS. Simply put, this involves you linking multiple AWS event-driven services together using Lambda to build your solution.


  • Your building a distributed system which will perform predictably and consistently at any scale
  • More predictable costing is possible
  • More optimal costing is likely
  • Dramatically simplified operations with a much lower operational burden
  • Reduced design and development time, less code required


  • The epitome of vendor lock-in
  • Not applicable to all types of problems
  • Less control and flexibility

Alright, let’s build something.

RPG Character Viewer Application – Overview

Let’s make an RPG character database, the purpose of the app will be to display, filter, and sort a list of character stats.

First, let’s start with an overview of the key services we will be using to build our serverless solution:

  1. Identity and Access Management (IAM)
    • Define roles and access policies for all service interaction
  2. CloudFront
    • Content delivery network with globally distributed edge locations
    • Works with both static and dynamic content, it will be serving all of our data
    • When users hit our system, they are hitting their nearest edge location and either pulling down a cached copy from that edge, or the edge makes an origin request back to AWS via their optimized private network
  3. API Gateway
    • Fully managed HTTP endpoints which invoke AWS events to call other services
    • When users are fetching data within the app, those API requests are hitting API Gateway.
  4. S3 Bucket
    • Fully managed object storage
    • This is where we would host our web application for user download (the URL users would navigate to in their browser)
  5. Route 53 DNS
    • Fully managed DNS
    • This will be the authoritative DNS resolver for all users of our service
  6. Lambda Functions
    • Event driven, stateless programs which run in a fully managed environment
    • When our APIs are invoked via API Gateway, they will call Lambda Functions we write to fetch the desired data from DyanmoDB
  7. DynamoDB
    • Fully managed NoSQL database
    • This will store our character stats and references to the characters images which are stored in S3

DynamoDB Data

Let’s create a DynamoDB table to contain the data about our characters, we will begin with just the character attributes for now.

Create characterData table

Create the characterData table by loading this template into CloudFormation, or by creating it manually in the AWS console, whichever:

Load some sample data

Implement Lambda Function for listCharacters API

Let’s create our Lambda function which will be responsible for querying data from DynamoDB when our listCharacters is invoked from API Gateway.

Steps in the AWS console:

  1. Create a new lambda function
    1. Author from scratch
      1. name: listCharacters
      2. Role: Create new role from template
        1. Role Name: listCharactersLambdaExecutionRole
        2. Policy template: Simple Microservices permissions
    2. Use the following source for your function definition and save it

Attach an access policy to your Lambda functions execution role

Each time the Lambda function executes, it assumes the role we associated with it during it’s creation: listCharactersLambdaExecutionRole. Out of the box, with the default template we selected that role only has permissions to write output to CloudWatch logs. We need to give that role enough access to our DynamoDB table so it can fetch the data.

You can attach the following custom policy to your listCharactersLambdaExecutionRole in IAM. IMPORTANT, you must update the arn string specifying the dynamoDB resources to be valid to your account, replace the xxxxxxxxxxxx placeholder with your AWS account id. Alternativly you can also generate this using the policy generator.

Test listCharacters function

In the AWS console you should see a “Test” button, here we can specify the payload of mock event that we can invoke our function with. The following is a generic template from an API Gateway event which I’ve updated to include our expected parameters:

View the results

Go into CloudWatch Logs, select your lambda function name(listCharacters)  log group and then select the most recent bundle. You should be able to see the console.log output from your function, including the character data from dynamoDB, sorted by character name in descending order.

If you are seeing an access denied error, it should give you the specific details. Most likely the cause is due to the access policy you attached to your lambda execution role, be sure you updated this to have your account id in the dynamoDB ARN as my policy included a placeholder: xxxxxxxxxxxxx.

Create API gateway resource backed by our Lambda function

In order to invoke our Lambda function from a web browser, we will expose it the internet via REST API endpoints managed by AWS: API Gateway. With APIGateway we can configure a resource and collection of HTTP methods to be backed by our Lambda function.

Perform the following configuration in the AWS Console:

  • APIGateway
    • APIs:
      • Create API
        • New API
          • API Name: characterViewerAPI
          • endpoint: edge optimized
        • *Click create*
      • characterViewerAPI
        • Resources
          • Actions dropdown
            • Create Method
              • GET
                • Integration Type: Lambda Function
                • Use Lambda Proxy Integration
                • Lambda Region: select region you are using
                • Lambda function: select your function
              • *Click save*

You should now be able to select the method you just created, now try to invoke it using the API Gateway test tool.

Test our API from an HTTP client or browser

Finally, we will deploy our API so that we can invoke it by hitting an internet exposed URL.

Perform the following configuration in the AWS Console:

  • APIGateway
    • characterViewerAPI
      • Resources
        • Actions dropdown
          • Deploy API
            • deployment stage: [new stage]
            • Stage name: test
              • *Click deploy*
          • Stages
            • test
            • *copy the invoke URL*

Using a tool such as postman or a web browser, send an HTTP GET request to the invoke URL. Include the following query string parameters in your request:

Key Value
sortfield strength-index
sortorder desc
classfilter warrior
resultLimit  5

The response payload should mirror what you saw you earlier when you tested via the Lambda mock invocation event.