Today I want to touch on a feature in AWS called Auto Scaling Groups – what is it, how it’s related to cost and how to control it
What are Auto Scaling Groups?
Auto Scaling Group is an advanced feature and is very useful when architecting a solution in AWS. The focus of this post is how cost is handled, so I won’t go too deep with the technicalities of Auto Scaling Group. However, I feel it’s important you’d understand the basics before proceeding.
Auto Scaling helps you ensure that you have the correct number of Amazon EC2 instances available to handle the load for your application. You create collections of EC2 instances, called Auto Scaling groups. What is Auto Scaling? (AWS)
Basically, it’s a feature that Amazon provides you on top of EC2 to control how many instances from a specific type you want up and running in any giving moment.
So, in it’s very nature, Auto Scaling Group (or ASG) looks at metrics you define (such as instance health check) and increase or decrease the number of instances in the group based on those metrics. Let me give an example to simplify:
You may define a policy stating that you want 3 instances of your web front-end server up in any giving moment. If one of the instances goes down for some reason (let’s say, a failure in an AWS Availability Zone), the Auto Scaling Group mechanism will kick in and spin up a new instance instead of the failed one.
So, just from the example above you can see how ASG can help you use only what you need. I suggest you read more on how to get started with ASG here and play a bit with the tool to understand it better, as there are a lot of configurations involved that are out of scope of this post
Last note, before I go into the actual use case – ASG on its own has no cost. The cost is the number of instances up and running as a result of the policy defined in the ASG.
The problem lies in the nature of the ASG – the default behavior is to launch and destroy EC2 instances based on the policy you define. But…
This means you have X number of instances UP in any giving moment. It also means you can’t STOP the instances, because the nature of ASG is to DESTROY the instance… Again, let’s give an example to illustrate a potential use case where this configuration can be a problem:
You run a development lab, and in this lab you have an auto scaling group for your web front end. You want to test your application with 3 web servers running simultaneously.
Now, since it’s a lab you don’t really need them up all the time, so at the end of the day you want to turn them off to save on spending… (Sidenote – one might argue that you should test the destroy and launch process as well while your building your solution, and I agree with that statement, but I find it to be a different use case, and both are valid in my mind…)
So… if you want to shut down your environment to keep costs down, but ASG destroys and creates new ones on the fly, what can you do???
Luckily, AWS has us covered. The secret lies in a feature called Auto Scaling Processes. You can suspend a specific Auto Scaling process manually to override the default behavior. Once suspended, the entire ASG configuration is still in place, except the process you explicitly set to suspend.
In our use case, we will suspend the terminate & launch processes, thus preventing the ASG mechanism from terminating our stopped instances or launching new ones instead of the stopped ones
OK, I get it. Show me how it’s done!
The long way…
Open up your AWS Console and go to the EC2 service.
Then, on the left pane click on Auto Scaling Groups and click on Auto Scaling Groups links in the main pane
Next, choose the group you wish to edit and in the bottom pane, in the details tab, click on edit:
Now comes the actual configuration – choose the processes you want to suspend. In our case, the processes will be launch & terminate. By choosing these you assure that once the instance is stopped, it won’t be terminated, and others won’t be up instead of the stopped instance:
Click on save, stop your instance and see the configuration in action (actually, you won’t see anything happens, and that’s our desired outcome)
(To revert the configuration, simply delete the Launch & Terminate processes in the box)
The short way
Prerequisite – You need a machine with AWS Cli tools installed. You can find the installation instructions here.
Once installed and configured, run the following command to list all your ASGs (replace the value for –region with the region you wish to query):
aws autoscaling describe-auto-scaling-groups --region us-east-1 --query 'AutoScalingGroups[*].AutoScalingGroupName' --output text
The result is a list with all the ASGs in the region. Identify the one you wish to modify and proceed to the next step
Once identified, run the following command on the desired ASG
aws autoscaling --region us-east-1 suspend-processes --auto-scaling-group-name myasg --scaling-processes Launch Terminate
Done! the ASG is configured.
Run this to resume the processes:
aws autoscaling --region us-east-1 resume-processes --auto-scaling-group-name myasg --scaling-processes Launch Terminate
The above suggested configuration can greatly help you keep your costs down –
- You can stop the instances once you don’t use them and as a result, you won’t pay for them
- New instances won’t launch, thus preventing unneeded charges for new instances
As mentioned in previous posts, same goes here – you can automate it using a scheduled task that can run daily at 7PM for example (using windows task scheduler or linux cron job, for example), or use other triggers and methods that you may be more familiar with (Lambda functions, CloudWatch metrics, etc.)
Be careful when and how to use it – in some cases (even most cases) you actually do want the ASG to bring up new instances and terminate unhealthy ones
The full documentation on Auto Scaling Processes from AWS can be found here