Terraform 12 Tutorial - AWS ASG and Terraform Modules
Here is the basic sample (main.tf):
provider "aws" {
region = "us-east-1"
}
resource "aws_instance" "busybox_web_server" {
ami = "ami-07ebfd5b3428b6f4d"
instance_type = "t2.nano"
vpc_security_group_ids = [aws_security_group.busybox.id]
user_data = <<-EOF
#!/bin/bash
echo "Hello, Terraform & AWS" > index.html
nohup busybox httpd -f -p "${var.http_port}" &
EOF
tags = {
Name = "busybox web server created via terraform"
}
}
resource "aws_security_group" "busybox" {
name = "terraform-busybox-sg"
ingress {
from_port = var.http_port
to_port = var.http_port
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
}
variable "http_port" {
description = "The port the sweb erver will be listening"
type = number
default = 8080
}
output "public_ip" {
value = aws_instance.busybox_web_server.public_ip
description = "The public IP of the web server"
}
There are couple of ways of set AWS secrets, we can use env variables:
export AWS_ACCESS_KEY_ID=AKI... export AWS_SECRET_ACCESS_KEY=oNw...
Run "terraform init", "terraform plan", and then "terraform apply":
$ terraform apply ... Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes aws_security_group.busybox: Creating... aws_security_group.busybox: Creation complete after 4s [id=sg-07afbcd7acfca30c5] aws_instance.busybox_web_server: Creating... aws_instance.busybox_web_server: Still creating... [10s elapsed] aws_instance.busybox_web_server: Still creating... [20s elapsed] aws_instance.busybox_web_server: Creation complete after 26s [id=i-05eed13622a27c697] Apply complete! Resources: 2 added, 0 changed, 0 destroyed. Outputs: public_ip = 52.90.97.222
Using the output information, we can check if it works:
$ curl http://52.90.97.222:8080 Hello, Terraform & AWS
The code is available from https://github.com/Einsteinish/AWS-Terraform-Introduction-Samples.git: basic branch
ASG helps us to automatically launch EC2 Instances, monitor their health, restart failed nodes, and adjust the size of the cluster in response to varying demands.
The first step using an ASG is to create a launch configuration, which specifies how to configure each EC2 Instance in the ASG. This will be replacing the instance configuration we used in the previous section.
We'll use create_before_destroy to create the replacement first, and then deleting the old one. Note that the default order is to delete the old resource and then create the new one.
Note that in the resource aws_autoscaling_group and in the aws_elb, we specified one more parameter, availability_zones using a data source.
Here is the modified code sample (main.tf):
provider "aws" {
region = "us-east-1"
}
variable "server_port" {
description = "The port the web server will be listening"
type = number
default = 8080
}
variable "elb_port" {
description = "The port the elb will be listening"
type = number
default = 80
}
data "aws_availability_zones" "all" {}
resource "aws_launch_configuration" "asg-launch-config-sample" {
image_id = "ami-07ebfd5b3428b6f4d"
instance_type = "t2.nano"
security_groups = [aws_security_group.busybox.id]
user_data = <<-EOF
#!/bin/bash
echo "Hello, Terraform & AWS ASG" > index.html
nohup busybox httpd -f -p "${var.server_port}" &
EOF
lifecycle {
create_before_destroy = true
}
}
resource "aws_security_group" "busybox" {
name = "terraform-busybox-sg"
ingress {
from_port = var.server_port
to_port = var.server_port
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
}
resource "aws_security_group" "elb-sg" {
name = "terraform-sample-elb-sg"
# Allow all outbound
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
# Inbound HTTP from anywhere
ingress {
from_port = var.elb_port
to_port = var.elb_port
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
}
resource "aws_autoscaling_group" "asg-sample" {
launch_configuration = aws_launch_configuration.asg-launch-config-sample.id
availability_zones = data.aws_availability_zones.all.names
min_size = 2
max_size = 5
load_balancers = [aws_elb.sample.name]
health_check_type = "ELB"
tag {
key = "Name"
value = "terraform-asg-sample"
propagate_at_launch = true
}
}
resource "aws_elb" "sample" {
name = "terraform-asg-sample"
security_groups = [aws_security_group.elb-sg.id]
availability_zones = data.aws_availability_zones.all.names
health_check {
target = "HTTP:${var.server_port}/"
interval = 30
timeout = 3
healthy_threshold = 2
unhealthy_threshold = 2
}
# Adding a listener for incoming HTTP requests.
listener {
lb_port = var.elb_port
lb_protocol = "http"
instance_port = var.server_port
instance_protocol = "http"
}
}
output "elb_dns_name" {
value = aws_elb.sample.dns_name
description = "The domain name of the load balancer"
}
In the code for aws_elb, we are telling the ELB to receive HTTP requests on port 80 which is the default port for HTTP and to route them to the port used by the server instances in the ASG. Note that, because, by default, ELB doesn't allow any incoming or outgoing traffic, we added a new security group to explicitly allow inbound requests on port 80 and all outbound requests.
Then, we need to tell the ELB to use aws_security_group, elb-sg by adding the security_groups parameter
This will create an ELB that will be deployed across all of the AZs where our multiple servers that can run in separate AZs. AWS will automatically scale the number of load balancer servers up and down based on traffic and handle failover if one of those servers goes down so that we can get scalability and high availability.
Note also that the aws_elb code above shows we're telling the ELB how to route requests by adding listeners which specify what port the ELB should listen on and what port it should route the request to.
Another feature of ELB is that it can periodically check the health of our EC2 Instances and, if an instance is unhealthy, it will automatically stop routing traffic to it. So, we added an HTTP health check where the ELB will send an HTTP request every 30 seconds to the /" URL of each of the EC2 Instances and only mark an Instance as healthy if it responds with a 200 OK.
Lastly, before deploying the load balancer, we need to add its DNS name as an output.
Run "terraform apply":
$ terraform apply ... Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes aws_elb.sample: Creating... aws_elb.sample: Still creating... [10s elapsed] aws_elb.sample: Creation complete after 13s [id=terraform-asg-sample] aws_autoscaling_group.asg-sample: Creating... aws_autoscaling_group.asg-sample: Still creating... [10s elapsed] aws_autoscaling_group.asg-sample: Still creating... [20s elapsed] aws_autoscaling_group.asg-sample: Still creating... [30s elapsed] aws_autoscaling_group.asg-sample: Still creating... [40s elapsed] aws_autoscaling_group.asg-sample: Creation complete after 44s [id=tf-asg-20200326175236084300000001] Apply complete! Resources: 2 added, 0 changed, 0 destroyed. Outputs: elb_dns_name = terraform-asg-sample-168878470.us-east-1.elb.amazonaws.com
Using the output information, we can check if it works:
$ curl http://terraform-asg-sample-168878470.us-east-1.elb.amazonaws.com Hello, Terraform & AWS ASG
The code is available from https://github.com/Einsteinish/AWS-Terraform-Introduction-Samples.git: asg branch
Clean up the resources:
$ terraform destroy ... aws_security_group.elb-sg: Destruction complete after 1m45s Destroy complete! Resources: 5 destroyed.
Though any Terraform configuration file in a folder is a module to see what modules are really capable of, we have to use one module from another module.
We'll break the main.tf into main.tf, variables.tf, and outputs.tf:
main.tf:
data "aws_availability_zones" "all" {}
resource "aws_launch_configuration" "asg-launch-config-sample" {
image_id = "ami-07ebfd5b3428b6f4d"
instance_type = "t2.nano"
security_groups = [aws_security_group.busybox.id]
user_data = <<-EOF
#!/bin/bash
echo "Hello, Terraform & AWS ASG" > index.html
nohup busybox httpd -f -p "${var.server_port}" &
EOF
lifecycle {
create_before_destroy = true
}
}
resource "aws_security_group" "busybox" {
name = "terraform-busybox-sg"
ingress {
from_port = var.server_port
to_port = var.server_port
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
}
resource "aws_security_group" "elb-sg" {
name = "terraform-sample-elb-sg"
# Allow all outbound
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
# Inbound HTTP from anywhere
ingress {
from_port = var.elb_port
to_port = var.elb_port
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
}
resource "aws_autoscaling_group" "asg-sample" {
launch_configuration = aws_launch_configuration.asg-launch-config-sample.id
availability_zones = data.aws_availability_zones.all.names
min_size = 2
max_size = 5
desired_capacity = 2
load_balancers = [aws_elb.sample.name]
health_check_type = "ELB"
tag {
key = "Name"
value = "terraform-asg-sample"
propagate_at_launch = true
}
}
resource "aws_elb" "sample" {
name = "terraform-asg-sample"
security_groups = [aws_security_group.elb-sg.id]
availability_zones = data.aws_availability_zones.all.names
health_check {
target = "HTTP:${var.server_port}/"
interval = 30
timeout = 3
healthy_threshold = 2
unhealthy_threshold = 2
}
# Adding a listener for incoming HTTP requests.
listener {
lb_port = var.elb_port
lb_protocol = "http"
instance_port = var.server_port
instance_protocol = "http"
}
}
variables.tf:
variable "server_port" {
description = "The port the web server will be listening"
type = number
default = 8080
}
variable "elb_port" {
description = "The port the elb will be listening"
type = number
default = 80
}
outputs.tf:
output "elb_dns_name" {
value = aws_elb.sample.dns_name
description = "The domain name of the load balancer"
}
$ mkdir -p modules/services/webservers mv *.tf modules/services/webservers
We can create a new main.tf for dev environment such as dev/services/webservers/main.tf with provider info and the Terraform code referring to the module:
provider "aws" {
region = "us-east-1"
}
module "webservers" {
source = "../../../modules/services/webservers"
}
The file structure should look like this:
. ├── README.md ├── dev │ └── services │ └── webservers │ └── main.tf ├── modules │ └── services │ └── webservers │ ├── main.tf │ ├── outputs.tf │ └── variables.tf ├── terraform.tfstate └── terraform.tfstate.backup
We can achieve the same as we did in the previous section:
$ cd dev/services/webservers $ terraform init Initializing modules... - webservers in ../../../modules/services/webservers ...
Note that with the setup it seems that we can reuse the module for "stage" and "prod" environemnts.
However, actually, if we run the apply command on this code, we get name conflict errors becases the in the webservers module, all the names are hard-coded. For example, the name of the security groups, ELB, and other resources are all hard-coded. To fix these issues, we need to add configurable inputs to the webservers module so it can behave differently in different environments.
modules/services/webservers/variables.tf:
...
variable "cluster_name" {
description = "The name to use for all the cluster resources"
type = string
}
modules/services/webservers/main.tf:
...
resource "aws_security_group" "busybox" {
name = "${var.cluster_name}-busybox-sg"
ingress {
from_port = var.server_port
to_port = var.server_port
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
}
resource "aws_security_group" "elb-sg" {
name = "${var.cluster_name}-elb-sg"
# Allow all outbound
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
# Inbound HTTP from anywhere
ingress {
from_port = var.elb_port
to_port = var.elb_port
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
}
resource "aws_autoscaling_group" "asg-sample" {
launch_configuration = aws_launch_configuration.asg-launch-config-sample.id
availability_zones = data.aws_availability_zones.all.names
min_size = 2
max_size = 5
desired_capacity = 2
load_balancers = [aws_elb.sample.name]
health_check_type = "ELB"
tag {
key = "Name"
value = "${var.cluster_name}-asg"
propagate_at_launch = true
}
}
resource "aws_elb" "sample" {
name = "${var.cluster_name}-asg-elb"
security_groups = [aws_security_group.elb-sg.id]
availability_zones = data.aws_availability_zones.all.names
health_check {
target = "HTTP:${var.server_port}/"
interval = 30
timeout = 3
healthy_threshold = 2
unhealthy_threshold = 2
}
# Adding a listener for incoming HTTP requests.
listener {
lb_port = var.elb_port
lb_protocol = "http"
instance_port = var.server_port
instance_protocol = "http"
}
}
Now, in the dev environment, in dev/services/webservers/main.tf, we can set the new input variable accordingly:
provider "aws" {
region = "us-east-1"
}
module "webservers" {
source = "../../../modules/services/webservers"
cluster_name = "webservers-dev"
}
We can check if that's working:
$ aws ec2 describe-security-groups --region us-east-1 |grep webservers
"GroupName": "webservers-dev-busybox-sg",
"GroupName": "webservers-dev-elb-sg",
$ aws elb describe-load-balancers --region us-east-1 |grep webservers
"CanonicalHostedZoneName": "webservers-dev-asg-elb-734091428.us-east-1.elb.amazonaws.com",
"DNSName": "webservers-dev-asg-elb-734091428.us-east-1.elb.amazonaws.com",
"LoadBalancerName": "webservers-dev-asg-elb",
"GroupName": "webservers-dev-elb-sg"
Note that when we run dev/services/webservers/main.tf, we're not getting output because it's not defined in the 'dev' env. We can get the output variables from the module by defining outputs.tf for the 'dev' environment:
output "elb_dns_name" {
value = module.webservers.elb_dns_name
description = "The domain name of the load balancer"
}
$ terraform output elb_dns_name = webservers-dev-asg-elb-734091428.us-east-1.elb.amazonaws.com
. ├── README.md ├── dev │ └── services │ └── webservers │ ├── main.tf │ ├── outputs.tf │ ├── terraform.tfstate │ └── terraform.tfstate.backup ├── modules │ └── services │ └── webservers │ ├── main.tf │ ├── outputs.tf │ └── variables.tf ├── terraform.tfstate └── terraform.tfstate.backup 6 directories, 10 files
The code is available from https://github.com/Einsteinish/AWS-Terraform-Introduction-Samples.git: modules branch
Suppose we want to use different EC2 instance_type or number of instances of ASG such as min_size/max_size/desired_capacity depending on the environments(dev/stage/prod). We need to set those variables in dev/services/webservers/main.tf:
provider "aws" {
region = "us-east-1"
}
module "webservers" {
source = "../../../modules/services/webservers"
cluster_name = "webservers-dev"
instance_type = "t2.nano"
min_size = 2
max_size = 5
desired_capacity = 2
}
We need to modify modules as well.
modules/services/webservers/variables.tf
variable "instance_type" {
description = "The type of EC2 Instances to run (e.g. t2.micro)"
type = string
}
variable "min_size" {
description = "The minimum number of EC2 Instances in the ASG"
type = number
}
variable "max_size" {
description = "The maximum number of EC2 Instances in the ASG"
type = number
}
variable "desired_capacity" {
description = "The desired number of EC2 Instances in the ASG"
type = number
}
modules/services/webservers/main.tf
...
resource "aws_autoscaling_group" "asg-sample" {
launch_configuration = aws_launch_configuration.asg-launch-config-sample.id
availability_zones = data.aws_availability_zones.all.names
min_size = var.min_size
max_size = var.max_size
desired_capacity = var.desired_capacity
load_balancers = [aws_elb.sample.name]
health_check_type = "ELB"
tag {
key = "Name"
value = "${var.cluster_name}-asg"
propagate_at_launch = true
}
}
...
$ terraform apply ... Outputs: elb_dns_name = webservers-dev-asg-elb-734091428.us-east-1.elb.amazonaws.com $ curl http://webservers-dev-asg-elb-734091428.us-east-1.elb.amazonaws.com Hello, Terraform & AWS ASG
The code is available from https://github.com/Einsteinish/AWS-Terraform-Introduction-Samples.git: modules-2 branch
For a "prod" environment, we may want to set scheduled ASG policy.
. ├── README.md ├── dev │ └── services │ └── webservers │ ├── main.tf │ ├── outputs.tf │ ├── terraform.tfstate │ └── terraform.tfstate.backup ├── modules │ └── services │ └── webservers │ ├── main.tf │ ├── outputs.tf │ └── variables.tf ├── prod │ └── services │ └── webservers │ ├── main.tf │ ├── outputs.tf │ ├── terraform.tfstate │ └── terraform.tfstate.backup ├── terraform.tfstate └── terraform.tfstate.backup
Here is our Scheduled ASG (/prod/services/webservers/main.tf):
provider "aws" {
region = "us-east-1"
}
module "webservers" {
source = "../../../modules/services/webservers"
cluster_name = "webservers-dev"
instance_type = "t2.nano"
min_size = 2
max_size = 5
desired_capacity = 2
}
# scale out - day
resource "aws_autoscaling_schedule" "scale_out_business_hours" {
scheduled_action_name = "scale-out-during-business-hours"
min_size = 2
max_size = 10
desired_capacity = 5
recurrence = "0 9 * * *"
}
# scale in - night
resource "aws_autoscaling_schedule" "scale_in_at_night" {
scheduled_action_name = "scale-in-at-night"
min_size = 2
max_size = 10
desired_capacity = 2
recurrence = "0 17 * * *"
}
However, note that aws_autoscaling_schedule are missing a required parameter, autoscaling_group_name which specifies the name of the ASG. The ASG itself is defined within the webserver module. We can access module output variables the same way as resource output attributes. The syntax is:
module.<MODULE_NAME>.<OUTPUT_NAME>
The outputs are defined in modules/services/webservers/outputs.tf:
...
output "asg_name" {
value = aws_autoscaling_group.asg-sample.name
description = "The name of the Auto Scaling Group"
}
So, our final Scheduled ASG (/prod/services/webservers/main.tf) should look like this:
prod/services/webservers/main.tf
provider "aws" {
region = "us-east-1"
}
module "webservers" {
source = "../../../modules/services/webservers"
cluster_name = "webservers-dev"
instance_type = "t2.nano"
min_size = 2
max_size = 5
desired_capacity = 2
}
# scale out - day
resource "aws_autoscaling_schedule" "scale_out_business_hours" {
scheduled_action_name = "scale-out-during-business-hours"
min_size = 2
max_size = 10
desired_capacity = 5
recurrence = "0 9 * * *"
autoscaling_group_name = module.webservers.asg_name
}
# scale in - night
resource "aws_autoscaling_schedule" "scale_in_at_night" {
scheduled_action_name = "scale-in-at-night"
min_size = 2
max_size = 10
desired_capacity = 2
recurrence = "0 17 * * *"
autoscaling_group_name = module.webservers.asg_name
}
$ terraform plan
...
Terraform will perform the following actions:
# aws_autoscaling_schedule.scale_in_at_night will be created
+ resource "aws_autoscaling_schedule" "scale_in_at_night" {
+ arn = (known after apply)
+ autoscaling_group_name = "tf-asg-20200326230230621900000002"
+ desired_capacity = 2
+ end_time = (known after apply)
+ id = (known after apply)
+ max_size = 10
+ min_size = 2
+ recurrence = "0 17 * * *"
+ scheduled_action_name = "scale-in-at-night"
+ start_time = (known after apply)
}
# aws_autoscaling_schedule.scale_out_business_hours will be created
+ resource "aws_autoscaling_schedule" "scale_out_business_hours" {
+ arn = (known after apply)
+ autoscaling_group_name = "tf-asg-20200326230230621900000002"
+ desired_capacity = 5
+ end_time = (known after apply)
+ id = (known after apply)
+ max_size = 10
+ min_size = 2
+ recurrence = "0 9 * * *"
+ scheduled_action_name = "scale-out-during-business-hours"
+ start_time = (known after apply)
}
...
As before, run terraform apply!
The code is available from https://github.com/Einsteinish/AWS-Terraform-Introduction-Samples.git: modules-3 branch
This blog is based on How to create reusable infrastructure with Terraform modules
Terraform
Ph.D. / Golden Gate Ave, San Francisco / Seoul National Univ / Carnegie Mellon / UC Berkeley / DevOps / Deep Learning / Visualization
My YouTube channel