Using CloudFormation To Build VPC's

This is part one of a two part post I am making with regards to CloudFormation and VPCs.  In this post we will construct a lean, yet usable VPC using a single CloudFormation script.  This is just a quick run through of creating and then tearing down a VPC via CloudFormation.  We'll concentrate on the basics and introduce the core concepts. In the second post we will dive deeper and expand the capabilities of our VPC CloudFormation script.  We will have it construct (among other things), jump boxes, NAT capability, and other enhanced security measures.

Why use CloudFormation

AWS CloudFormation brings to reality the statement: Infrastructure As Code.  It is a JSON DSL format that allows us to define and maintain the complete manifestation of our software system's infrastructure and even application servers within source controlled documents.  This brings to DevOps and Operations the benefits of control and change tracking that have long realized in software development.

I strongly suggest that if you are going to leverage AWS services, start from the beginning with CloudFormation.  Not only will you gain the benefits described above,  in leveraging CloudFormation, you will gain quite a few insights as to how AWS resources are constructed and how those services intercommunicate.  In a sense, you are working with the cloud equivalent of bare metal. With regards to AWS, there is really no lower layer than the network.  In the current AWS universe, this is manifest as a VPC (Virtual Private CLoud).  This article describes how to fully define and launch a VPC.

Prerequisites for this exercise

  1. Basic knowledge of the purpose of VPCs is expected.  No prior experience with CloudFormation is required.
  2. You will need an AWS account.  If you do not have one, you can get one here and qualify for 1 year of free tier.
  3. Download the Cloudformation template, vpc-lean.template.json to your computer.

A note about AWS Regions

Select a Region with at least 3 Availability Zones

Select a Region with at least 3 Availability Zones

The top level of geographic distinction in AWS is what is know as a Region. Any given Cloudformation stack is created in a Region (all the resources that stack creates will be in the region selected).

For these exercises, we need Regions with at least 3 Availability Zones.  If you are in the US, I'd suggest using Oregon or N. Virginia (N. California only has 2 regions so cannot be used)

 

Launch the lean VPC stack

This is a VPC and Subnets only stack that you can start with.  There is no cost to launch this stack so it is good way to see how cloudformation works in both creating and tearing down stacks.

For this Cloudformation stack we will use the AWS console to create it (in part two we'll switch to using the commandline). 
So in a browser go to the AWS CloudFormation console at:  https://console.aws.amazon.com/cloudformation/home Select a Region with at least 3 Availability Zones as metioned above and click the Create Stack button in the upper left. That will bring you to this screen:

Select Template Screen

Select Template Screen

Under the Choose a template section, click the Browse... button and select the vpc-lean.template.json you downloaded above.  Then click the Next button.

Note Template Storage: Behind the scenes, AWS will place the template you select into an S3 bucket it creates on your account.  You'll see the path to this bucket in the Review screen under the heading Template URL with a value something akin to: https://s3-us-west-2.amazonaws.com/cf-templates-1ol6s5vaspjfr-us-west-2/2016080bpV-vpc-lean.template.json.

You will then be on the Specify Details screen.  On this screen, simply provide a name for your CloudFormation stack, then choose an EnvName (used to build the VPC name), and then select 3 TargetAZs (these will auto populate dynamically based on the region you are creating the CloudFormation stack within).

Specify Details Screen

Specify Details Screen

Click the Next button and you will be on the Options screen.  For this exercise you can skip that screen by simply clicking the Next button.

You are now on the Review screen where you are shown all the choices made on previous screens.  You can return to those screens and make changes if you like but here we will simply click the Create button.  We will then be brought to the CloudFormation stack list screen for the given Region.  You will see the Status of the stack you are creating listed as CREATE_IN_PROGRESS.  There is a row of tabs below the list of stacks (Overview, Outputs, Resources, ...) where you can expose various aspects of the stack selected in the list.  

Stack Create In Progress

Stack Create In Progress

Creation time varies somewhat, but the stack construction will take on average about 2 minutes to complete.  The status of the stack will transition to CREATE_COMPLETE.  You can check the resources tab and see the impressive list of what this CloudFormation stack has accomplished in about 2 minutes.

Stack Create Complete

Stack Create Complete

You now have a fully functional VPC, with 3 public and 3 private subnets spread across 3 Availability zones.  You can view it in the VPC section of the AWS console (Click Services -> VPC).  Feel free to explore the created resources but at this point, don't change anything in the console.  Changes to the CloudFormation created VPC resources are typically done via pushing updates to the CloudFormation stack, but that's a topic for future post.  

One Click Cleanup

Return to the CloudFormation stack list, and select the stack we've just created.  Then under Actions, select Delete Stack.

Select Delete Stack

Select Delete Stack

You will then see the stack enter the DELETE_IN_PROGRESS phase.

Deleting Stack Events

Deleting Stack Events

Once the Deletion process completes, the stack will simply disappear from the stack list and all resources that it had created will be gone. 

Summary

This is a pragmatic approach to getting familiar with CloudFormation since if you are utilizing AWS in any way, you are most likely leveraging VPC so best to know how to build them in repeatable fashion.  You can easily adapt this VPC template to build more or less subnets, change the names given to the VPC's, etc by making changes to the template.  There is certainly room for improvement in this VPC.  In order to follow some AWS best practice such as, only placing ELBs in public subnets and application servers in private subnets, we will need some additional VPC functionality, namely, Jump (aka Bastion) hosts and NAT capability.  In part two of this post, we will accomplish this by defining these additional resources in our CloudFormation template, thus keeping the entirety of the definition of our VPC within source controlled CloudFormation template.