Terraform Tutorial - Using Terraform Registry AWS Modules

In this post, we will use modules from the Terraform Registry for AWS.

When calling a module:

1. The source argument is required. Terraform will search for a module in the Terraform registry that matches the given string. We could also use a URL or local file path for the source of the modules. After adding, removing, or modifying module blocks, we must re-run terraform init to allow Terraform the opportunity to adjust the installed modules.


2. The other argument is the version. For supported sources, the version will let us define which version or versions of the module will be loaded. By default terraform init command will not upgrade an already-installed module; use the -upgrade option to instead upgrade to the newest available version. When using modules installed from a module registry, it is recommend to constrain explicitly the acceptable version numbers to avoid unexpected or unwanted changes as shown below:

   module "consul" {
     source  = "hashicorp/consul/aws"
     version = "0.0.5"
   
     servers = 3
   }    
   



3. Other arguments to module blocks are treated as input variables to the modules.

Every Terraform configuration has at least one module (root module), which consists of the resources defined in the .tf files in the main working directory.

A module can call other modules, which lets us include the child module's resources into the configuration.

Modules are called from within other modules using module blocks:

module "servers" {
  source = "./app-cluster"

  servers = 5
}    

A module that includes a module block like this is the calling module of the child module.

The resources defined in a module are encapsulated, so the calling module cannot access their attributes directly. However, the child module can declare output values to selectively export certain values to be accessed by the calling module.

For example, if the ./app-cluster module referenced in the example above exported an output value named instance_ids then the calling module can reference that result using the expression module.servers.instance_ids:

resource "aws_elb" "example" {
  # ...

  instances = module.servers.instance_ids
}    

The Terraform Registry includes a directory of ready-to-use modules for various common purposes, which can serve as larger building-blocks for our infrastructure.

The following code (main.tf) is using modules from the Terraform registry.

├── LICENSE
├── README.md
├── main.tf
├── modules
│   └── aws-s3-static-website-bucket
│       ├── LICENSE
│       ├── README.md
│       ├── main.tf
│       ├── outputs.tf
│       ├── variables.tf
│       └── www
│           ├── error.html
│           └── index.html
├── outputs.tf
└── variables.tf    

Here are the files.

main.tf:

# Terraform configuration

provider "aws" {
  region = "us-east-1"
}

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "2.21.0"

  name = var.vpc_name
  cidr = var.vpc_cidr

  azs             = var.vpc_azs
  private_subnets = var.vpc_private_subnets
  public_subnets  = var.vpc_public_subnets

  enable_nat_gateway = var.vpc_enable_nat_gateway

  tags = var.vpc_tags
}

module "ec2_instances" {
  source  = "terraform-aws-modules/ec2-instance/aws"
  version = "2.12.0"

  name           = "my-ec2-cluster"
  instance_count = 2

  ami                    = "ami-0c5204531f799e0c6"
  instance_type          = "t2.micro"
  vpc_security_group_ids = [module.vpc.default_security_group_id]
  subnet_id              = module.vpc.public_subnets[0]

  tags = {
    Terraform   = "true"
    Environment = "dev"
  }
}

module "website_s3_bucket" {
  source = "./modules/aws-s3-static-website-bucket"

  bucket_name = "bogo-test-april-19-2021"

  tags = {
    Terraform   = "true"
    Environment = "dev"
  }
}  

outputs.tf:

# Output variable definitions

output "vpc_public_subnets" {
  description = "IDs of the VPC's public subnets"
  value       = module.vpc.public_subnets
}

output "ec2_instance_public_ips" {
  description = "Public IP addresses of EC2 instances"
  value       = module.ec2_instances.public_ip
}

output "website_bucket_arn" {
  description = "ARN of the bucket"
  value       = module.website_s3_bucket.arn
}

output "website_bucket_name" {
  description = "Name (id) of the bucket"
  value       = module.website_s3_bucket.name
}

output "website_bucket_domain" {
  description = "Domain name of the bucket"
  value       = module.website_s3_bucket.domain
} 

variables.tf:

# Input variable definitions

variable "vpc_name" {
  description = "Name of VPC"
  type        = string
  default     = "example-vpc"
}

variable "vpc_cidr" {
  description = "CIDR block for VPC"
  type        = string
  default     = "10.0.0.0/16"
}

variable "vpc_azs" {
  description = "Availability zones for VPC"
  type        = list(string)
  default     = ["us-east-1a", "us-east-1b", "us-east-1c"]
}

variable "vpc_private_subnets" {
  description = "Private subnets for VPC"
  type        = list(string)
  default     = ["10.0.1.0/24", "10.0.2.0/24"]
}

variable "vpc_public_subnets" {
  description = "Public subnets for VPC"
  type        = list(string)
  default     = ["10.0.101.0/24", "10.0.102.0/24"]
}

variable "vpc_enable_nat_gateway" {
  description = "Enable NAT gateway for VPC"
  type        = bool
  default     = true
}

variable "vpc_tags" {
  description = "Tags to apply to resources created by VPC module"
  type        = map(string)
  default = {
    Terraform   = "true"
    Environment = "dev"
  }
}

modules/aws-s3-static-website-bucket/main.tf:

resource "aws_s3_bucket" "s3_bucket" {
  bucket = var.bucket_name

  acl    = "public-read"
  policy = <<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PublicReadGetObject",
            "Effect": "Allow",
            "Principal": "*",
            "Action": [
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::${var.bucket_name}/*"
            ]
        }
    ]
}
EOF

  website {
    index_document = "index.html"
    error_document = "error.html"
  }

  tags = var.tags
}  

modules/aws-s3-static-website-bucket/outputs.tf:

# Output variable definitions

output "arn" {
  description = "ARN of the bucket"
  value       = aws_s3_bucket.s3_bucket.arn
}

output "name" {
  description = "Name (id) of the bucket"
  value       = aws_s3_bucket.s3_bucket.id
}

output "domain" {
  description = "Domain name of the bucket"
  value       = aws_s3_bucket.s3_bucket.website_domain
} 

modules/aws-s3-static-website-bucket/variables.tf:

# Input variable definitions

variable "bucket_name" {
  description = "Name of the s3 bucket. Must be unique."
  type        = string
}

variable "tags" {
  description = "Tags to set on the bucket."
  type        = map(string)
  default     = {}
}

creates IAM user with a random password, a pair of IAM access/secret keys and uploads IAM SSH public key. User password and secret key is encrypted using public key of keybase.io user named bogo_user_1

main.tf:

# Terraform configuration

terraform {
  required_version = ">= 0.12.6"

  required_providers {
    aws = ">= 2.50"
  }
}

provider "aws" {
  region = "us-east-1"
}

#########################################
# IAM user, login profile and access key
#########################################
module "iam_user_1" {
  # source = "../../modules/iam-user"
  source  = "terraform-aws-modules/iam/aws//modules/iam-user"

  name          = "bogo_user_1"
  force_destroy = true

  # User "bogotobogo" has uploaded his public key here - https://keybase.io/test/pgp_keys.asc
  pgp_key = "keybase:bogotobogo"

  password_reset_required = false

  # SSH public key
  upload_iam_user_ssh_key = true

  ssh_public_key = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC82CMkjBEO2c/Bbp4vJojTCkjSk6n66uKC5f1jay+OTlqbyDy2EuVs23w0/3xh0iX+8K1rmFt9C4I3EWbjz5A+DZuop4TH/sBHgI2h6mIklGbJ6rL3QqMHeedm5dS2RrlgsBuc0woJwP8VVqB4tnHsSEDf9YYPkJ80eD/+PaoMMhvqaH9aDLhQuzlGtco5U8bL2lmIIqXV+KNowImFxuYGzCP8TNKxgxf5EbMpLXFDSTQ0h8CVGcLxRvW7NI8TbvCraRKFvhYFdM2fbnxJDOLEcHazYfu/R2cn4JvUsTWtKYzNq2vsL2LqigI2OiEt8piITdsphnuh7rM+x2MkU2/t kihyuckhong@Kihyucks-Air.attlocal.net"
}

###################################################################
# IAM user without pgp_key (IAM access secret will be unencrypted)
###################################################################
module "iam_user_2" {
  # source = "../../modules/iam-user"
  source  = "terraform-aws-modules/iam/aws//modules/iam-user"

  name = "bogo_user_2"

  create_iam_user_login_profile = false
  create_iam_access_key         = true
}    

outputs.tf:

# module.iam_user_1

output "iam_user_name" {
  description = "The user's name"
  value       = module.iam_user_1.iam_user_name
}

output "iam_user_arn" {
  description = "The ARN assigned by AWS for this user"
  value       = module.iam_user_1.iam_user_arn
}

output "iam_user_unique_id" {
  description = "The unique ID assigned by AWS"
  value       = module.iam_user_1.iam_user_unique_id
}

output "iam_user_login_profile_key_fingerprint" {
  description = "The fingerprint of the PGP key used to encrypt the password"
  value       = module.iam_user_1.iam_user_login_profile_key_fingerprint
}

output "iam_user_login_profile_encrypted_password" {
  description = "The encrypted password, base64 encoded"
  value       = module.iam_user_1.iam_user_login_profile_encrypted_password
}

output "iam_access_key_id" {
  description = "The access key ID"
  value       = module.iam_user_1.iam_access_key_id
}

output "iam_access_key_key_fingerprint" {
  description = "The fingerprint of the PGP key used to encrypt the secret"
  value       = module.iam_user_1.iam_access_key_key_fingerprint
}

output "iam_access_key_encrypted_secret" {
  description = "The encrypted secret, base64 encoded"
  value       = module.iam_user_1.iam_access_key_encrypted_secret
}

output "iam_access_key_secret" {
  description = "The access key secret"
  value       = module.iam_user_1.iam_access_key_secret
  sensitive   = true
}

output "iam_access_key_ses_smtp_password_v4" {
  description = "The secret access key converted into an SES SMTP password"
  value       = module.iam_user_1.iam_access_key_ses_smtp_password_v4
  sensitive   = true
}

output "iam_access_key_status" {
  description = "Active or Inactive. Keys are initially active, but can be made inactive by other means."
  value       = module.iam_user_1.iam_access_key_status
}

output "pgp_key" {
  description = "PGP key used to encrypt sensitive data for this user (if empty - secrets are not encrypted)"
  value       = module.iam_user_1.pgp_key
}

output "keybase_password_decrypt_command" {
  description = "Decrypt user password command"
  value       = module.iam_user_1.keybase_password_decrypt_command
}

output "keybase_password_pgp_message" {
  description = "Encrypted password"
  value       = module.iam_user_1.keybase_password_pgp_message
}

output "keybase_secret_key_decrypt_command" {
  description = "Decrypt access secret key command"
  value       = module.iam_user_1.keybase_secret_key_decrypt_command
}

output "keybase_secret_key_pgp_message" {
  description = "Encrypted access secret key"
  value       = module.iam_user_1.keybase_secret_key_pgp_message
}


# module.iam_user_2

output "iam_user_name_2" {
  description = "The user's name"
  value       = module.iam_user_2.iam_user_name
}

output "iam_user_arn_2" {
  description = "The ARN assigned by AWS for this user"
  value       = module.iam_user_2.iam_user_arn
}

output "iam_user_unique_id_2" {
  description = "The unique ID assigned by AWS"
  value       = module.iam_user_2.iam_user_unique_id
}

output "iam_user_login_profile_key_fingerprint_2" {
  description = "The fingerprint of the PGP key used to encrypt the password"
  value       = module.iam_user_2.iam_user_login_profile_key_fingerprint
}

output "iam_user_login_profile_encrypted_password_2" {
  description = "The encrypted password, base64 encoded"
  value       = module.iam_user_2.iam_user_login_profile_encrypted_password
}

output "iam_access_key_id_2" {
  description = "The access key ID"
  value       = module.iam_user_2.iam_access_key_id
}

output "iam_access_key_key_fingerprint_2" {
  description = "The fingerprint of the PGP key used to encrypt the secret"
  value       = module.iam_user_2.iam_access_key_key_fingerprint
}

output "iam_access_key_encrypted_secret_2" {
  description = "The encrypted secret, base64 encoded"
  value       = module.iam_user_2.iam_access_key_encrypted_secret
}

output "iam_access_key_secret_2" {
  description = "The access key secret"
  value       = module.iam_user_2.iam_access_key_secret
  sensitive   = true
}

output "iam_access_key_ses_smtp_password_v4_2" {
  description = "The secret access key converted into an SES SMTP password"
  value       = module.iam_user_2.iam_access_key_ses_smtp_password_v4
  sensitive   = true
}

output "iam_access_key_status_2" {
  description = "Active or Inactive. Keys are initially active, but can be made inactive by other means."
  value       = module.iam_user_2.iam_access_key_status
}

output "pgp_key_2" {
  description = "PGP key used to encrypt sensitive data for this user (if empty - secrets are not encrypted)"
  value       = module.iam_user_2.pgp_key
}

output "keybase_password_decrypt_command_2" {
  description = "Decrypt user password command"
  value       = module.iam_user_2.keybase_password_decrypt_command
}

output "keybase_password_pgp_message_2" {
  description = "Encrypted password"
  value       = module.iam_user_2.keybase_password_pgp_message
}

output "keybase_secret_key_decrypt_command_2" {
  description = "Decrypt access secret key command"
  value       = module.iam_user_2.keybase_secret_key_decrypt_command
}

output "keybase_secret_key_pgp_message_2" {
  description = "Encrypted access secret key"
  value       = module.iam_user_2.keybase_secret_key_pgp_message
}    

$ terraform apply --auto-approve
module.iam_user_2.aws_iam_user.this[0]: Creating...
module.iam_user_1.aws_iam_user.this[0]: Creating...
module.iam_user_2.aws_iam_user.this[0]: Creation complete after 1s [id=bogo_user_2]
module.iam_user_1.aws_iam_user.this[0]: Creation complete after 1s [id=bogo_user_1]
module.iam_user_2.aws_iam_access_key.this_no_pgp[0]: Creating...
module.iam_user_1.aws_iam_access_key.this[0]: Creating...
module.iam_user_1.aws_iam_user_login_profile.this[0]: Creating...
module.iam_user_1.aws_iam_user_ssh_key.this[0]: Creating...
module.iam_user_2.aws_iam_access_key.this_no_pgp[0]: Creation complete after 0s [id=AKIAQV57YS3YGD65F7RT]
module.iam_user_1.aws_iam_user_login_profile.this[0]: Creation complete after 1s [id=bogo_user_1]
module.iam_user_1.aws_iam_access_key.this[0]: Creation complete after 1s [id=AKIAQV57YS3YNHKEYHH5]
module.iam_user_1.aws_iam_user_ssh_key.this[0]: Creation complete after 1s [id=APKAQV57YS3YGG6UUPNH]

Apply complete! Resources: 6 added, 0 changed, 0 destroyed.

Outputs:

iam_access_key_encrypted_secret = "wcFMA7YvMLjoVTeDy...QbSTivTWN6uEsYwA="
iam_access_key_encrypted_secret_2 = ""
iam_access_key_id = "AKIAQV57YS3YNHKEYHH5"
iam_access_key_id_2 = "AKIAQV57YS3YGD65F7RT"
iam_access_key_key_fingerprint = "c5a5eb34cca30a3d8734c36f54bdf021ced7ec95"
iam_access_key_key_fingerprint_2 = ""
iam_access_key_secret = <sensitive>
iam_access_key_secret_2 = <sensitive>
iam_access_key_ses_smtp_password_v4 = <sensitive>
iam_access_key_ses_smtp_password_v4_2 = <sensitive>
iam_access_key_status = "Active"
iam_access_key_status_2 = "Active"
iam_user_arn = "arn:aws:iam::047109936880:user/bogo_user_1"
iam_user_arn_2 = "arn:aws:iam::047109936880:user/bogo_user_2"
iam_user_login_profile_encrypted_password = "wcFMA7YvMLj...F4loOn4PhPD4A"
iam_user_login_profile_encrypted_password_2 = ""
iam_user_login_profile_key_fingerprint = "c5a5eb34cca30a3d8734c36f54bdf021ced7ec95"
iam_user_login_profile_key_fingerprint_2 = ""
iam_user_name = "bogo_user_1"
iam_user_name_2 = "bogo_user_2"
iam_user_unique_id = "AIDAQV57YS3YM6IPCL3Z4"
iam_user_unique_id_2 = "AIDAQV57YS3YCDRNE55D5"
keybase_password_decrypt_command = <<EOT
echo "wcFMA7...loOn4PhPD4A" | base64 --decode | keybase pgp decrypt

EOT
keybase_password_pgp_message = <<EOT
-----BEGIN PGP MESSAGE-----
Version: Keybase OpenPGP v2.0.76
Comment: https://keybase.io/crypto

wcFMA7YvVzDAKo1zj7jWal...PhPD4A
-----END PGP MESSAGE-----

EOT
keybase_secret_key_decrypt_command = <<EOT
echo "wcFM...YwA=" | base64 --decode | keybase pgp decrypt

EOT
keybase_secret_key_pgp_message = <<EOT
-----BEGIN PGP MESSAGE-----
Version: Keybase OpenPGP v2.0.76
Comment: https://keybase.io/crypto

wcFMA7YvM...uEsYwA=
-----END PGP MESSAGE-----

EOT
pgp_key = "keybase:bogotobogo"
pgp_key_2 = ""

The following configuration creates IAM group with users who are allowed to assume IAM roles.

Note: Modules in Package Sub-directories

When the source of a module is a version control repository or archive file (generically, a "package"), the module itself may be in a sub-directory relative to the root of the package. A special double-slash(//) syntax is interpreted by Terraform to indicate that the remaining path after that point is a sub-directory within the package.

hashicorp/consul/aws//modules/consul-cluster    

References:

1. Use Modules from the Registry
2. https://registry.terraform.io/search/modules?namespace=terraform-aws-modules
3. Terraform module which creates IAM resources on AWS
4. Github - IAM user example

Terraform