Down the Cloud Formation rabbit-hole


In my previous post I explained how to deploy the basic compute resources that will run our applications. If you haven’t done so already, please read this first

I got some questions from readers that wanted to understand a bit more about the cloudformation tool I’m using. I can see how it might be a bit overwhelming at first, so I want to demystify some of the “magic” that is done behind the scenes

In this post, I want to dive a bit deeper to a key feature inside the cloud formation template – the userdata piece.

EC2 Instance Userdata

What is it?

If you know it already or not, you can “inject” user data into an ec2 instance when you first launch it.

This is done only once, when the instance is launched for the first timeIt won’t happen every restart, just on the initial creation of the instance.

You can use it in several different ways, all are described in detail here.

I added a shell script to the CloudFormation template I used in my previous post, to configure multiple settings on the EC2 instances.

The complete, updated CloudFormation template can be found here (you might notice I converted it from .json to .yaml. I find yaml to be much easier to read, and they both achieve the same goal)

Deep Dive

Let’s take a deep look at the relevant piece of code that holds the user data:

    - ''
    - - |
      - |
        export PATH=/usr/local/bin:$PATH;
      - |
        curl --silent --location | bash -
      - |
        yum update -y
      - |
        yum install fio git nodejs docker -y
      - |
        fio --filename=/dev/xvdm --rw=read --bs=128k --iodepth=32 --ioengine=libaio --direct=1 --name=volume-initialize
      - |
        mkfs -t ext4 /dev/xvdm
      - |
        mkdir /data
      - |
        cp /etc/fstab /home/ec2-user/fstab.bak
      - |
        echo '/dev/xvdm /data ext4 defaults,nofail 0 2' | tee -a /etc/fstab
      - |
        mount -a
      - |
        chmod 777 -R /data
      - |
        service docker start
      - |
        usermod -a -G docker ec2-user
      - |
        curl -L`uname -s`-`uname -m` > /usr/local/bin/docker-compose
      - |
        chmod +x /usr/local/bin/docker-compose
      - |
        chown root:docker /usr/local/bin/docker-compose
      - |
        mkdir -p /etc/systemd/system/docker.service.d
      - |
        cat <<EOF > /home/ec2-user/docker.conf
      - |
      - |
      - |
        ExecStart=/usr/bin/dockerd -H unix:///var/run/docker.sock -H tcp://
      - |
      - |
        mv /home/ec2-user/docker.conf /etc/systemd/system/docker.service.d/docker.conf
      - |
        systemctl daemon-reload
      - |
        systemctl restart docker
      - |
        systemctl status docker

What does it do?

This is basically a shell script, just formatted slightly different to fit the cloudformation yaml syntax:

  • installing fio, git, nodejs & docker
  • creating a 50gb partition on an ebs volume
  • enabling docker remote api (you can read more details about it in a previous blog post)


I hope you see now that this is a great process to bootstrap your deployment. But we all know that things don’t always go as expected. And since our machine is in the cloud, how can we troubleshoot it’s boot process?

Glad you asked.

The entire boot process is visible via the AWS management console and the AWS-CLI. In order to see it via the web console, simply choose your instance in the EC2 console and click on Actions > Instance Settings > Get System Log:


Alternatively (and preferably, if you ask me) you can run a command using AWS-CLI:

aws ec2 get-console-output --instance-id {yourinstanceid} --output text

Further reading about console output can be found here.

If you need guidance with installing and configuring the aws cli you can read all about it here


So there you have it, a simple way to bootstrap your ec2 instance startup configuration and keep it consistent when launched. Now that you know how it works, what do you think should be added?

You can expect to see more and more layers of the cloud formation template uncovered in future posts. stay tuned

always take the red pill π


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s