Using Terraform to Create a Linux Image with Cloud Watch Agent and Amazon Agent

In this installment of my adventures with Terraform.  I want to spin up an Amazon Linux AMI with the Cloud Watch Agent and the Amazon Agent.

The Cloud Watch Agent will configure the ability to send the logs for the EC2 Instance to Cloud Watch.

The Amazon Agent will enable Amazon Inspector to inspect the instance for security vulnerabilities.

The first step is to create a Terraform script that remotely executes a shell script.

Three parts are required.
Connection
This provides the ssh connection info in order to connect to the EC2 Instance.
  connection {
    user        = "${var.INSTANCE_USERNAME}"
    private_key = "${file("${var.PATH_TO_PRIVATE_KEY}")}"
  }
File Provisioner 
This allows us to take an arbitraty files and upload it to the EC2 instance, we could just put all of the scripts in the remote-exec, but that would be ugly.
  provisioner "file" {
    source      = "awscli.conf"
    destination = "/tmp/awscli.conf"
  }

  provisioner "remote-exec" {
    inline = [
      "chmod +x /tmp/script.sh",
      "sudo /tmp/script.sh",
      "sudo cp /tmp/awscli.conf /etc/awslogs/awscli.conf",
    ]
  }
Remote Exec Provisioner
Create a remote ssh session for executing commands on the EC2 Instance.
provisioner "remote-exec" {
    inline = [
      "chmod +x /tmp/script.sh",
      "sudo /tmp/script.sh",
      "sudo cp /tmp/awscli.conf /etc/awslogs/awscli.conf",
    ]
  }
So clearly there is some magic missing.  I mean what am i actually deploying?
For that we go to the Shell Script
First part install the Logging Agent
sudo yum update -y
sudo yum install -y awslogs
sudo service awslogs start
sudo chkconfig awslogs on
Here are the amazon instructions
Second part install the amazon agents
wget https://d1wk0tztpsntt1.cloudfront.net/linux/latest/install
sudo bash install
sudo /etc/init.d/awsagent start
sudo /opt/aws/awsagent/bin/awsagent status
Here are the amazon instructions
When you run terraform apply you get a lot of output.
Now we can jump into amazon inspector, use a filter for a tag and see what we find.

aws_instance.linuxec2 (remote-exec): Total download size: 78 k
aws_instance.linuxec2 (remote-exec): Installed size: 240 k
aws_instance.linuxec2 (remote-exec): Downloading packages:
aws_instance.linuxec2 (remote-exec): (1/2): aws-cli-p |  69 kB     00:00
aws_instance.linuxec2 (remote-exec): (2/2): awslogs-1 | 8.8 kB     00:00
aws_instance.linuxec2 (remote-exec): ----------------------------------------
aws_instance.linuxec2 (remote-exec): Total      436 kB/s |  78 kB  00:00
aws_instance.linuxec2 (remote-exec): Running transaction check
aws_instance.linuxec2 (remote-exec): Running transaction test
aws_instance.linuxec2 (remote-exec): Transaction test succeeded
aws_instance.linuxec2 (remote-exec): Running transaction
aws_instance.linuxec2 (remote-exec):   Installing : aws-cli- [         ] 1/2
aws_instance.linuxec2 (remote-exec):   Installing : aws-cli- [#        ] 1/2
aws_instance.linuxec2 (remote-exec):   Installing : aws-cli- [####     ] 1/2
aws_instance.linuxec2 (remote-exec):   Installing : aws-cli- [#####    ] 1/2
aws_instance.linuxec2 (remote-exec):   Installing : aws-cli- [######   ] 1/2
aws_instance.linuxec2 (remote-exec):   Installing : aws-cli- [######## ] 1/2
aws_instance.linuxec2 (remote-exec):   Installing : aws-cli-plugin-clo   1/2
aws_instance.linuxec2 (remote-exec):   Installing : awslogs- [         ] 2/2
aws_instance.linuxec2 (remote-exec):   Installing : awslogs- [#####    ] 2/2
aws_instance.linuxec2 (remote-exec):   Installing : awslogs- [######   ] 2/2
aws_instance.linuxec2 (remote-exec):   Installing : awslogs- [######## ] 2/2
aws_instance.linuxec2 (remote-exec):   Installing : awslogs-1.1.2-1.10   2/2
aws_instance.linuxec2 (remote-exec):   Verifying  : awslogs-1.1.2-1.10   1/2
aws_instance.linuxec2 (remote-exec):   Verifying  : aws-cli-plugin-clo   2/2

aws_instance.linuxec2 (remote-exec): Installed:
aws_instance.linuxec2 (remote-exec):   awslogs.noarch 0:1.1.2-1.10.amzn1



aws_instance.linuxec2 (remote-exec): Transaction Summary
aws_instance.linuxec2 (remote-exec): ========================================
aws_instance.linuxec2 (remote-exec): Install  1 Package

aws_instance.linuxec2 (remote-exec): Total size: 5.9 M
aws_instance.linuxec2 (remote-exec): Installed size: 5.9 M
aws_instance.linuxec2 (remote-exec): Downloading packages:
aws_instance.linuxec2 (remote-exec): Running transaction check
aws_instance.linuxec2 (remote-exec): Running transaction test
aws_instance.linuxec2 (remote-exec): Transaction test succeeded
aws_instance.linuxec2 (remote-exec): Running transaction
aws_instance.linuxec2 (remote-exec):   Installing : AwsAgent [         ] 1/1
aws_instance.linuxec2 (remote-exec):   Installing : AwsAgent [#        ] 1/1
aws_instance.linuxec2 (remote-exec):   Installing : AwsAgent [##       ] 1/1
aws_instance.linuxec2 (remote-exec):   Installing : AwsAgent [###      ] 1/1
aws_instance.linuxec2 (remote-exec):   Installing : AwsAgent [####     ] 1/1
aws_instance.linuxec2 (remote-exec):   Installing : AwsAgent [#####    ] 1/1
aws_instance.linuxec2 (remote-exec):   Installing : AwsAgent [######   ] 1/1
aws_instance.linuxec2 (remote-exec):   Installing : AwsAgent [#######  ] 1/1
aws_instance.linuxec2 (remote-exec):   Installing : AwsAgent [######## ] 1/1
aws_instance.linuxec2 (remote-exec):   Installing : AwsAgentKernelModu   1/1
aws_instance.linuxec2 (remote-exec):   Verifying  : AwsAgentKernelModu   1/1

aws_instance.linuxec2 (remote-exec): Installed:
aws_instance.linuxec2 (remote-exec):   AwsAgentKernelModule__amzn__4.4.41-36.55.amzn1.x86_64 0:1.0.27.1-0

Here is a link to the github repo with the working code.
Look in the folder ec2instances.

Deploying to AWS using Terraform for Local Experimentation

Terraform is a tool used to turn an api into code, and in this case we are going to use with AWS.

The download of terraform is available here

You don’t want aws secrets in your git repo, so this is a basic structure i start out with when i am doing local terraform experimentation.

Terraform Setup on Ubuntu

There is a nice script here that installs terraform.  Just run it periodically to keep it up to date.

https://github.com/ryanmaclean/awful-bash-scripts/blob/master/install_recent_terraform_packer.sh

I am a noob in the linux stuff so i needed to google to how to make it executable.

sudo chmod +x scriptname

vars.tf

Used to define the set of variables used in Terraform

variable "AWS_ACCESS_KEY" { }
variable "AWS_SECRET_KEY" { }
variable "AWS_REGION" 
{
    default = "us-east-1"
}

 

provider.tf

Defines what api will get ran. Since this is aws we are deploying to we are using the aws provider.  If we were deploying to azure the provider would be azurerm

provider "aws" {
    access_key = "${var.AWS_ACCESS_KEY}"
    secret_key = "${var.AWS_SECRET_KEY}"
    region = "${var.AWS_REGION}
}

terraform.tfvars

Put this in the git ignore so the secrets stay on your local machine. If we were using a build server an alternative solution would need to be used.  But for a small scripting effort this works great.

AWS_ACCESS_KEY = ""
AWS_SECRET_KEY = ""
AWS_REGION = ""

instance.tf

This one can really be named anything. It is where all of the resources would get built out.

resource "aws_instance" "example" {
   ami             = ""
   instance_type   = "t2.micro"
}

Here is the github repo that this is stored in.

https://github.com/jonshern/terraformstarter