Using Using Lambda-backed custom resources to extend the functionality of CloudFormation

This is part 3 of a 3 part series that I wrote on using CloudFormation to rollout a personal VPN internet proxy. This part guides you through the details of extending the functionality of CloudFormation using a Lambda-backed custom resource to clean the secrets bucket on stack deletion. You can find the article posted here on Cloud Assessments:

How to roll your own VPN with AWS CloudFormation – Part three

Corresponding video walkthrough that you can find here on YouTube:

The article will guide you through the content of this template:

Automate the installation and configuration of OpenVPN with CloudFormation

This is part 2 of a 3 part series that I wrote on using CloudFormation to rollout a personal VPN internet proxy. Part 2 walks you through the details of configuring OpenVPN as a personal internet proxy and automating it with CloudFormation. You can find the article posted here on Cloud Assessments:

How to roll your own VPN with AWS CloudFormation – Part two

Corresponding video walkthrough that you can find here on YouTube:

The article will guide you through the content of this template:

Automate the creation of a basic VPC with CloudFormation

This is part 1 of a 3 part series that I wrote on using CloudFormation to rollout a personal VPN internet proxy. Part 1 walks you through the details of the VPC creation, leaving you with a template that could be a basis for other simple projects. You can find the article posted here on Cloud Assessments:

How to roll your own VPN with AWS CloudFormation – Part one

Corresponding video walkthrough that you can find here on YouTube:

The article will guide you through the content of this template:

 

Let’s build a serverless web app on AWS

Summary

It’s possible to build a web service using an application platform that is fully managed by AWS. Simply put, this involves you linking multiple AWS event-driven services together using Lambda to build your solution.

Advantages:

  • Your building a distributed system which will perform predictably and consistently at any scale
  • More predictable costing is possible
  • More optimal costing is likely
  • Dramatically simplified operations with a much lower operational burden
  • Reduced design and development time, less code required

Disadvantages:

  • The epitome of vendor lock-in
  • Not applicable to all types of problems
  • Less control and flexibility

Alright, let’s build something.

RPG Character Viewer Application – Overview

Let’s make an RPG character database, the purpose of the app will be to display, filter, and sort a list of character stats.


First, let’s start with an overview of the key services we will be using to build our serverless solution:

  1. Identity and Access Management (IAM)
    • Define roles and access policies for all service interaction
  2. CloudFront
    • Content delivery network with globally distributed edge locations
    • Works with both static and dynamic content, it will be serving all of our data
    • When users hit our system, they are hitting their nearest edge location and either pulling down a cached copy from that edge, or the edge makes an origin request back to AWS via their optimized private network
  3. API Gateway
    • Fully managed HTTP endpoints which invoke AWS events to call other services
    • When users are fetching data within the app, those API requests are hitting API Gateway.
  4. S3 Bucket
    • Fully managed object storage
    • This is where we would host our web application for user download (the URL users would navigate to in their browser)
  5. Route 53 DNS
    • Fully managed DNS
    • This will be the authoritative DNS resolver for all users of our service
  6. Lambda Functions
    • Event driven, stateless programs which run in a fully managed environment
    • When our APIs are invoked via API Gateway, they will call Lambda Functions we write to fetch the desired data from DyanmoDB
  7. DynamoDB
    • Fully managed NoSQL database
    • This will store our character stats and references to the characters images which are stored in S3

DynamoDB Data

Let’s create a DynamoDB table to contain the data about our characters, we will begin with just the character attributes for now.

Create characterData table

Create the characterData table by loading this template into CloudFormation, or by creating it manually in the AWS console, whichever:

Load some sample data

Implement Lambda Function for listCharacters API

Let’s create our Lambda function which will be responsible for querying data from DynamoDB when our listCharacters is invoked from API Gateway.

Steps in the AWS console:

  1. Create a new lambda function
    1. Author from scratch
      1. name: listCharacters
      2. Role: Create new role from template
        1. Role Name: listCharactersLambdaExecutionRole
        2. Policy template: Simple Microservices permissions
    2. Use the following source for your function definition and save it

Attach an access policy to your Lambda functions execution role

Each time the Lambda function executes, it assumes the role we associated with it during it’s creation: listCharactersLambdaExecutionRole. Out of the box, with the default template we selected that role only has permissions to write output to CloudWatch logs. We need to give that role enough access to our DynamoDB table so it can fetch the data.

You can attach the following custom policy to your listCharactersLambdaExecutionRole in IAM. IMPORTANT, you must update the arn string specifying the dynamoDB resources to be valid to your account, replace the xxxxxxxxxxxx placeholder with your AWS account id. Alternativly you can also generate this using the policy generator.

Test listCharacters function

In the AWS console you should see a “Test” button, here we can specify the payload of mock event that we can invoke our function with. The following is a generic template from an API Gateway event which I’ve updated to include our expected parameters:

View the results

Go into CloudWatch Logs, select your lambda function name(listCharacters)  log group and then select the most recent bundle. You should be able to see the console.log output from your function, including the character data from dynamoDB, sorted by character name in descending order.

If you are seeing an access denied error, it should give you the specific details. Most likely the cause is due to the access policy you attached to your lambda execution role, be sure you updated this to have your account id in the dynamoDB ARN as my policy included a placeholder: xxxxxxxxxxxxx.

Create API gateway resource backed by our Lambda function

In order to invoke our Lambda function from a web browser, we will expose it the internet via REST API endpoints managed by AWS: API Gateway. With APIGateway we can configure a resource and collection of HTTP methods to be backed by our Lambda function.

Perform the following configuration in the AWS Console:

  • APIGateway
    • APIs:
      • Create API
        • New API
          • API Name: characterViewerAPI
          • endpoint: edge optimized
        • *Click create*
      • characterViewerAPI
        • Resources
          • Actions dropdown
            • Create Method
              • GET
                • Integration Type: Lambda Function
                • Use Lambda Proxy Integration
                • Lambda Region: select region you are using
                • Lambda function: select your function
              • *Click save*

You should now be able to select the method you just created, now try to invoke it using the API Gateway test tool.

Test our API from an HTTP client or browser

Finally, we will deploy our API so that we can invoke it by hitting an internet exposed URL.

Perform the following configuration in the AWS Console:

  • APIGateway
    • characterViewerAPI
      • Resources
        • Actions dropdown
          • Deploy API
            • deployment stage: [new stage]
            • Stage name: test
              • *Click deploy*
          • Stages
            • test
            • *copy the invoke URL*

Using a tool such as postman or a web browser, send an HTTP GET request to the invoke URL. Include the following query string parameters in your request:

Key Value
sortfield strength-index
sortorder desc
classfilter warrior
resultLimit  5

The response payload should mirror what you saw you earlier when you tested via the Lambda mock invocation event.

 

Thoughts after completing all 7 AWS Certifications

Overall I had a great experience preparing for these, I enjoyed the journey and learned quite a bit.

Solutions Architect - Associate
Click to view

SysOps Administrator - Associate
Click to view

Developer - Associate
Click to view

Big Data - Specialty
Click to view

Advanced Networking - Specialty
Click to view

DevOps Engineer - Professional
Click to view

Solutions Architect - Professional
Click to view


This is the order I took the exams:

  1. Solutions Architect Associate
  2. Developer Associate
  3. SysOps Associate
  4. BigData Specialty
  5. Advanced Networking Specialty
  6. Solutions Architect Professional
  7. DevOps Professional

This is how I would rank them by difficulty, from easy to hard:

  1. Developer Associate (easy)
  2. Solutions Architect Associate
  3. SysOps Associate
  4. DevOps Professional
  5. BigData Specialty
  6. Advanced Networking Specialty
  7. Solutions Architect Professional (difficult)

There is a huge difference in difficulty between the associate and professional level exams.

I felt that Solutions Architect Professional was the most difficult, primarily due to the time constraint and the amount of information that you need to consider per question.

Determining readiness for taking an exam:

I spent approximately this amount of time preparing for each exam to cover the material outlined in the AWS exam guide:

  • Developer Associate – 20 hours
  • Solutions Architect Associate – 20 hours
  • SysOps Associate – 20 hours
  • BigData Speciality – 80 hours
  • Advanced Networking – 80 hours
  • DevOps Professional – 40 hours
  • Solutions Architect Professional – 160 hours

There is some content overlap and I found preparing for the specialty exams prior to SA PRO very beneficial because of this. That is also why I didn’t spend nearly as much time preparing for DevOps relatively speaking.

Once I felt like I had a grasp of the required material for each exam, I did the following to assess gaps in my knowledge:

Associate level:
  • acloud.guru mock exam and quizzes
Specialty level:
  • acloud.guru mock exam and quizzes
  • Linux Academy mock exam and quizzes
  • The official sample test questions from AWS
Professional Level:
  • acloud.guru mock exam and quizzes
  • Linux Academy mock exam and quizzes
  • The official sample test questions from AWS
  • The official online practice test from AWS

Learning content and strategy:

Associate Level:
  • acloud.guru course for certification
  • Reviewed high level documentation for most AWS services
  • Reviewed the core services more in depth
  • Definitely spend time preparing for these exams, however, don’t go overboard as they are a mile wide and an inch deep. If you can comfortably handle the LA or ACG mock exams (take both if your unsure), you will probably do well on the real thing.
Speciality and Professional Level:
  • YouTube – AWS Re:Invent Talks
    • Most of these are very high quality, especially at the 300/400 level.
    • Search YouTube for AWS Re:invent deep dive or 300/400 level for any aspect of AWS that you are interested in.
    • I treated these as audio content that I would listen to during my commutes, walks, runs, etc
  • For any video/audio content:
    • Rather than skip something entirely, use 1.5x – 2.5x playback speed when you encounter material you feel partially comfortable with.
  • Build some mini-projects of your own creation
    • Rather than systematically reading through reams and reams of AWS documentation, I would recommend building some mini/toy projects on AWS to drive your research. Not only is this way more fun and potentially useful, but I also find it a far better method of internalizing details.
  • Pick a few course labs, and instead of following along, build a CloudFormation template that achieves the same outcome.
    • For aspects of AWS that you don’t have much experience with, I found this really helpful because it requires you to understand more detail about the individual resource properties and their relationships. I think it’s easier to gloss over these subtleties when your doing everything from within the AWS console.
  • AWS Whitepapers, Guides, and External Reading
    • In general try to focus on content that cover topics you cannot easily get meaningful hands on experience with, such as:
      • BGP route propagation and VPC route priority
      • Direct Connect and VPN
      • Redshift
      • EMR, Hadoop, Spark
      • Active Directory integrations in hybrid cloud/onprem networks
      • DNS in hybrid cloud/onprem networks
      • VPC peering strategies
      • Multi-account access strategies
      • Deployment strategies
      • Disaster recovery strategies
      • Migration strategies
      • Patterns for high availability

Highlighted free content from my notes:

Certification YouTube Playlists Good Reads
 BigData
Networking
 DevOps Pro
 SA Pro

Using AWS as a personal on-demand VPN solution

I have been using AWS as a personal VPN solution for awhile, but I was doing so with semi-automation + a few manual steps. I decided to make a CloudFormation template to fully automate the provisioning, making it easy to only pay for the resources when you actually need to use them.

These are the steps taken by the template:

  • Create a new VPC, create a single subnet with a default route to the internet via VPC attached IGW
  • Create a new instance: t2.micro using the 2017.09 Amazon Linux AMI. As the actual AMI id differs per region, I included a static map keyed by region
  • An elastic network interface is created and a new elastic IP is associated
  • Within the EC2 instance, cfn-init handles all the bootstrapping:
    • easyrsa is installed and used to generate a ca and keys
    • openssl is used to generate a static tls key
    • An OpenVPN .ovpn client profile is generated from the concatenation of the key + cert + ca
    • openvpn server is installed and configured
  • An S3 bucket is created and the static tls key + client profile are uploaded
  • A Lambda function is created with a cfn-init event listener, when the delete stack event fires the lambda function will empty the S3 bucket and then delete it. The purpose of this Lambda function is just for this cleanup purpose as CloudFormation will not delete a bucket containing objects on it’s own.
  • A client user with access to the S3 bucket can download the profile, load into their OpenVPN client, and connect.
  • When your finished, all created resources are removed on stack deletion

Using it is simple, just run it in CloudFormation, download the ovpn client profile from S3 and your good to go. I embedded everything into a single template so that it would be as easy as possible.

You can download it here:

https://github.com/zugdud/openvpn_aws_cloudformation

P.S. If you get a missing key error while connecting, it is probably because you moved the profile out of the directory after extracting it. (The profile looks for the key in the same directory, which is included in the zip file)

Leverage CloudFront CDN with any HTTPS origin content

CloudFront can be used as a whole site CDN, including as a front for applications which serve a mix of semi-static and 0 TTL dynamic content. This can be a nice solution for interactive legacy applications, for example, something like a message board.

Consider these benefits:

  • AWS hosted origins only – All edge to origin requests traverse the optimized AWS private backbone network.
  • Any origin – When you restrict access to your origin to only CloudFront, you are insulating your origin services from DDoS attacks. This can be accomplished by injecting a pre-shared secret into the request header at the edge and rejecting requests without this value.
  • Any origin – When a viewer requests expired cache content from an edge location, the edge will not just fetch the origin content, it will first perform an HTTP HEAD request to check for changes. Only if the content differs from the cached copy will an origin GET be performed.
  • Any origin – CloudFront respects the origins cache-control HTTP header value to specify cache lifetime for the object, or optionally override them at the edge.
  • Any origin – Lambda at the edge! One useful use case for this is as an API adapter in situations where you do not want to make changes on the origin service. So rather than have to setup and manage proxy servers to transform requests, you can do this via CloudFront.

WordPress example

Let’s consider an example, you have just setup a new single EC2 instance on AWS hosting WordPress and want to leverage CloudFront. You want your domain apex: wastedbandwidth.org, to point viewers to your CloudFront distribution, backed by your WordPress origin server. To simplify the steps, you used Amazon as your domain registrar though that’s not a requirement.

Obviously this is not an optimal configuration as it makes assumptions to be generic.

Create Route53 DNS records

  1. CNAME record pointing to your origin server: origin.wastedbandwidth.org
  2. A record (Alias) pointing to your CloudFront distribution: wastedbandwidth.org

 

Configuring the origin server

  1. Enable SSL/TLS on the server using Let’s encrypt provides
  2. Install certbot to automate the certificate provisioning and configuration,
  3. Configure WordPress within wp-admin setting the site to be https://wastedbandwidth.org

Optional – if you do not have mail services for your new domain you can configure email forwarding using SES. 

When you request a new certificate using AWS certificate manager, confirmation emails will be sent to multiple email addresses associated with that domain. If don’t have a mail service for your domain, you can quickly setup an email forwarder using SES to receive the mail, fire an SNS event to invoke lambda, and script a simple handler in Lambda to process/store/forward it.

Create a CloudFront Web Distribution

 

Distribution Configuration

  1. Specify the AWS managed certificate that for wastedbandwidth.org that you requested earlier

Origin Configuration

  1. Specify your origin server: origin.wastedbandwidth.org
  2. Origin Protocol: 
  3. Origin Protocol Policy: 
  4. HTTPS Port: 443

Default Cache Behavior Settings

  1. Path Pattern: Default (*)
  2. Viewer Protocol Policy:
  3. Allowed HTTP Methods:  
  4. Whitelist Headers: Origin
  5. Object Caching:
    1. Minimum TTL 0
    2. Maximum TTL 0
    3. Default TTL 0
  6. Forward Cookies: All
  7. Query String Forwarding and Caching: Forward All

restrict origin requests to cloudfront

With the above, users/bots/attackers could access your site via CloudFront or directly using the origin.domain.com. For added security and optimization, you can restrict access to your origin by configuring cloudFront to inject a pre-shared secret as a header value. You can add these custom injected header values into your origin configuration settings. Your origin can then reject all traffic without this header.