-->

aws ,  devops cloudformation

If you have a habit creating your AWS resources manually, things can get very messy very quickly. At some point you realize that you have no idea if a resource is still in use and just to be safe you leave it alone.

I found myself in this situation and decided to take advantage Infrastructure as Code paradigm using AWS CloudFormation. To start simple I decided to migrate my automated CV response application to a CloudFormation stack. Going forward this is a much efficient way to write blog posts too. Instead of writing step by step instructions I can simply post the CloudFormation stack in JSON or YAML.

Infrastructure of Code in a nutshell

Basically this approach allows you to define, manage and provision all the resources that define your system.

Advantages:

  • Changes can be source-controlled
  • Entire provisioning process can be automated
  • entire infrastructure can be easily recreated in a different account.
  • Resources can easily be identified. Tags can be used to identify which stack the resources belong to.
  • Resources can easily be clean up. Deleting a stack will delete all the resource it created.

Basic Terminology

  • Stack: All the resources used to create an infrastructure.
  • StackSet: A StackSet is a container for AWS CloudFormation stacks that lets you provision stacks across AWS accounts and regions by using a single AWS CloudFormation template.
  • Design template: This is the file YAML or JSON format that defines all the resources that will be created AWS

Where to start…

Even though you get familiar with the concepts finding where tostart can be intimidating sometimes. When it comes to CloudFormation, there are a lot of sample templates that you can start and build upon.

So here’s an easy way to get started:

Step 1: Save the following snippet to a local file such as cloudformation.sample.yaml:

Resources:
  Ec2Instance:
    Type: AWS::EC2::Instance
    Properties:
      InstanceType: t2.micro
      ImageId: ami-0aff88ed576e63e90

In the example above I’m using a stock AWS Linux AMI in London region.

Step 2: Go to Stack and click Create Stack (Make sure you’re in EU London region otherwise AWS won’t be able to find the AMI specified in our template)

Step 3: In specify template section, select Upload a template file

Step 4: Clock Choose file to locate your file and upload your template.

Step 5: Click next and specify a stack name something like FirstStackForEC2 and click Next

Step 6: Click Next on Configure Stack Options view and in the final review step of the wizard click Create Stack.

At this point you can observe the progress of your stack being created.

Now if you go to EC2 service in the same region you should be able to see the new instance:

If you delete the stack, it will in turn delete everything it created which is the EC2 instance in this example.

Launch Stack URLs

I always like the launch stack buttons that I see every now and then. I think there’s something magical about clicking a button and watching an entire infrastructure being created right before your eyes!

Basically clicking the Launch Stack URL opens the wizard we just used with the fields populated with the values in the template.

Step 1: Upload the template to S3 and make it publicly accessible.

Step 2: Use the following naming convention and replace the values:

https://console.aws.amazon.com/cloudformation/home?region=region#/stacks/new?stackName=stack_name&templateURL=template_location

In this example I uploaded the Launch Stack button image to my GitHub repository so that I can link to it.

Resources

aws ,  devops cloudwatch, custom_metric

I had an issue recently with an EC2 instance running out of disk space. Unfortunately free disk space is not a metric that comes out of the box with AWS CloudWatch. This post is about implementing a custom metric and getting notifications via AWs CloudWatch based on that metric.

Steps to monitor disk space with CloudWatch

Step 1: Download sample config file

AWS provides a sample JSON file at this location: https://s3.amazonaws.com/ec2-downloads-windows/CloudWatchConfig/AWS.EC2.Windows.CloudWatch.json

Download a copy of this file.

Step 2: Set IsEnabled to true

By default it comes disabled so set the value as shown below:

"IsEnabled": true

Step 3: Add the custom metric for disk usage

Add the custom metric to monitor disk space:

{
    "Id": "PerformanceCounterDisk",
    "FullName": "AWS.EC2.Windows.CloudWatch.PerformanceCounterComponent.PerformanceCounterInputComponent,AWS.EC2.Windows.CloudWatch",
    "Parameters": {
        "CategoryName": "LogicalDisk",
        "CounterName": "% Free Space",
        "InstanceName": "C:",
        "MetricName": "FreeDiskPercentage",
        "Unit": "Percent",
        "DimensionName": "InstanceId",
        "DimensionValue": "{instance_id}"
    }
}

Step 4: Add the new metric to flows

After defining the metric we need to add it to the flows so that it can be sent to CloudWatch. To achieve this update the flows section as shown below:

"Flows": {
    "Flows": 
    [
        "(ApplicationEventLog,SystemEventLog),CloudWatchLogs",
        "(PerformanceCounter,PerformanceCounterDisk),CloudWatch"
    ]
}

Step 5: Add IAM role to server

It’s a good practice to manage permissions of EC2 instances via IAM roles assigned to the machine. To enable sending logs to CloudWatch add AmazonEC2RoleForSSM policy to the machine’s role

Without this role SSM agent service gets an access denied error.

Step 6: Restart Amazon SSM Agent service

Either by using Windows Services Manager or running the following command:

Restart-Service AmazonSSMAgent

Once this is all done wait a few minutes and check CloudWatch metrics. Under All -> Windows/Default you should be able to see new metric under InstanceId group (as that’s what we are using to group the logs). And when you click the metric you should be able to see a nice time-based graph of free disk space on the server:

Notes

  • It’s useful to know where SSM Agent’s logs are stored. They can be found in this path:

    %PROGRAMDATA%\Amazon\SSM\Logs\

  • The service reports every 5 minutes. The PollInterval in the JSON file is in seconds and is different than service report interval.

Resources

aws aurora, rds, serverless, database

A few months ago AWS announced a serverless model for their Aurora databases. Compared to traditional DB approach this is brand new.

I’ve been trying it out for a pilot application and it works well in general. You pay for what you use just like any other serverless resource.

The only problem I’ve been having is DB startup time after pause. Meaning after 5 minutes the resources are released and the first request that comes after that suffers a performance penalty. My application was getting an error when this happened and it was showing an error screen. Obviously from a user standpoint it’s not a great experience.

So to remedy this issue I’ve updated the DB connection timeout

Connection Timeout=120

By default it’s 15 seconds which is not enough for the new server to respond. But after increasing the timeout at least I could prevent the application from failing. Of course this doesn’t speed up the response time of the DB server.

They recently announced additional regions that support serverless Aurora.

For cost-cutting reasons this can be a great option. Especially if your system is idle for extended periods of time you don’t need to pay anything. Also it scales up so you don’t have to worry about the database bottlenecks under heavy traffic.

Resources