AWS Auto Scaling with Terraform: From Reactive to Predictive
Predictive scaling, Graviton and Spot capacity in Terraform, with configs that work, the metric to scale on, and how to measure the saving before you promise it.
By VVVHQ Team ·
Reactive auto scaling has a built-in delay. A CloudWatch alarm fires once load is already high, new instances take minutes to boot and pass health checks, and users feel slow responses in between. Teams cover the delay by running more instances than they need, and pay for that headroom around the clock.
Predictive scaling, Graviton instances and Spot capacity each go after a different part of that cost, and Terraform keeps the setup reviewable. This guide shows the configuration and how to build the cost case from your own numbers.
A mixed-instance Auto Scaling group in Terraform
A mixed instances policy lets one Auto Scaling group (ASG) combine Graviton (arm64) and x86 instance types, and On-Demand with Spot capacity. Arm and x86 need different AMIs, so each architecture gets its own launch template:
resource "aws_autoscaling_group" "app" {
name = "app-production"
min_size = 2
max_size = 20
vpc_zone_identifier = var.private_subnet_ids
target_group_arns = [aws_lb_target_group.app.arn]
health_check_type = "ELB"
mixed_instances_policy {
instances_distribution {
on_demand_base_capacity = 2
on_demand_percentage_above_base_capacity = 25
spot_allocation_strategy = "price-capacity-optimized"
}
launch_template {
launch_template_specification {
launch_template_id = aws_launch_template.app_arm64.id
version = aws_launch_template.app_arm64.latest_version
}
override {
instance_type = "m7g.xlarge" # Graviton3
}
override {
instance_type = "m6g.xlarge" # Graviton2
}
override {
instance_type = "m6i.xlarge" # x86 fallback, with its own AMI
launch_template_specification {
launch_template_id = aws_launch_template.app_x86.id
version = aws_launch_template.app_x86.latest_version
}
}
}
}
tag {
key = "Environment"
value = "production"
propagate_at_launch = true
}
}
desired_capacity is left out on purpose. If it is set, every terraform apply resets the group to that number and overrides what the scaling policies decided. The application, and its container images or packages, must be built for both architectures.
The price-capacity-optimized strategy picks Spot pools that combine low interruption risk with low price, and it is the strategy AWS recommends for most workloads.
On price, AWS quotes up to 40% better price performance for Graviton2 instances than for comparable x86 instances. Treat that as a ceiling and benchmark your own service on both before you change the order of preference. AWS prices Spot capacity at up to 90% below On-Demand, with a two-minute warning before an interruption, so keep Spot for stateless instances that can stop at short notice.
Three kinds of scaling policy
Target tracking keeps a metric near a target value, and it is the right default for most web services:
resource "aws_autoscaling_policy" "requests_per_target" {
name = "requests-per-target"
autoscaling_group_name = aws_autoscaling_group.app.name
policy_type = "TargetTrackingScaling"
target_tracking_configuration {
predefined_metric_specification {
predefined_metric_type = "ALBRequestCountPerTarget"
resource_label = "${aws_lb.app.arn_suffix}/${aws_lb_target_group.app.arn_suffix}"
}
target_value = 1000 # requests per instance; take the number from a load test
}
}
Step scaling reacts by different amounts depending on how far an alarm threshold is breached. Use it when you want fast scale-out during spikes and slower scale-in.
Predictive scaling forecasts load from history and launches capacity ahead of it. It runs alongside the other two, and when both apply, the group keeps the higher capacity.
Predictive scaling
Predictive scaling needs at least 24 hours of metric history and uses up to 14 days of it to forecast the next 48 hours. It suits services with a daily or weekly cycle, such as business-hours traffic or a nightly batch. For irregular, spiky load the forecast adds little.
resource "aws_autoscaling_policy" "predictive" {
name = "predictive-scaling"
autoscaling_group_name = aws_autoscaling_group.app.name
policy_type = "PredictiveScaling"
predictive_scaling_configuration {
mode = "ForecastOnly" # switch to ForecastAndScale once the forecast tracks reality
scheduling_buffer_time = 300
max_capacity_breach_behavior = "HonorMaxCapacity"
metric_specification {
target_value = 1000
predefined_metric_pair_specification {
predefined_metric_type = "ALBRequestCount"
resource_label = "${aws_lb.app.arn_suffix}/${aws_lb_target_group.app.arn_suffix}"
}
}
}
}
Start in ForecastOnly mode and compare the forecast with actual load for a week or two before you let it launch instances. A scheduling_buffer_time of 300 launches instances five minutes before the forecast says they are needed, which has to cover boot time, health checks and application warm-up. Measure yours and adjust.
Choosing the scaling metric
CPU is the default, and often a weak signal. A service waiting on a database or a downstream API can be slow at low CPU, and adding instances may make the database problem worse. Target tracking assumes that adding capacity brings the metric down, so pick a metric that moves in proportion to the number of instances:
- Requests per target behind a load balancer.
- Queue backlog per instance for workers that read from SQS.
- Concurrent connections per instance for WebSocket or long-polling services.
Latency is a poor target-tracking metric for the same reason: it does not fall in proportion to instance count, and when the bottleneck is elsewhere, more instances will not bring it down. Alert on p99 latency and scale on load.
Queue depth also lets a worker fleet go all the way to zero. On a medical-device platform we ran for four and a half years, the processing services took their jobs from SQS. We wrote small Lambda-based scalers keyed on queue depth, so the processing fleet and its ECS services scale down to zero when the queue is empty and come back when work arrives, without dropping jobs that are already in flight.
Karpenter on EKS
On EKS, Karpenter is the alternative to Cluster Autoscaler. Cluster Autoscaler resizes node groups you defined in advance, while Karpenter launches individual nodes chosen for the pods waiting to be scheduled. That gives you right-sized nodes from a wide set of instance types, consolidation that moves pods onto fewer nodes and removes empty ones when load drops, Spot with On-Demand fallback, and NodePools that prefer arm64 where your images support it. EKS Auto Mode runs Karpenter for you if you would rather not operate it.
Consolidation moves pods, so set PodDisruptionBudgets for services that cannot lose replicas at an arbitrary moment.
Making the cost case
The size of each saving depends on how your estate runs today, so build the case from your own bill instead of industry percentages:
| Lever | Where the saving comes from | Effort | What to check first | |---|---|---|---| | Graviton | Lower price per instance at similar performance | Medium | Arm builds of the app and its dependencies | | Spot above a base | Spot discount on stateless capacity | Low | Graceful shutdown within two minutes | | Predictive scaling | Less standing headroom | Low | A daily or weekly traffic cycle | | Karpenter | Better bin-packing on EKS | Medium | PodDisruptionBudgets, node startup time | | Better scaling metrics | Fewer unnecessary scale-outs | Medium | Instrumentation for the metric |
For each lever, take the current spend of the groups it applies to, run the change on a slice of production for a few weeks, and extrapolate from the measured result. The case you take to the board then rests on numbers from your own systems.
Where to start
- List every ASG with its monthly cost and its scaling policy, and flag those with simple scaling or none.
- Add predictive scaling in forecast-only mode to the most expensive groups that have a regular cycle.
- Build arm64 images and add Graviton types to the mixed instances policy, in non-production first.
- Add Spot capacity above the On-Demand base for stateless services.
- Replace CPU with request or queue metrics where CPU does not track load.
Each step stands on its own and can be measured before the next one starts.
We do this as a fixed-scope cost assessment or as part of a larger project, with the changes landing in your own Terraform. Describe how your groups scale today, and we will say which of these levers we would try first.