AWS : ECS with cloudformation and json task definition

bogotobogo.com site search:

Pre-run : cloudformation for VPC, ServiceDiscovery etc.

The following template (myECSVPC.yaml) will setup VPC (10.1.0.0/16) including subnets, InternetGateway, and Route tables), ECS ServiceDiscovery (private dns in Route53), autoscaling group including launch configuration for ECS instances, load balancer (NLB), EIPs, and Roles etc.

AWSTemplateFormatVersion: '2010-09-09'
Description: AWS CloudFormation template to create a new VPC or use an existing VPC
  for ECS deployment

Parameters:
  VPCCIDR:
    Description: CIDR Block for VPC
    Type: String
    Default: 10.1.0.0/16
    AllowedValues:
      - 10.1.0.0/16
  PublicSubnet1Param:
    Description: Public Subnet 1
    Type: String
    Default: 10.1.11.0/24
    AllowedValues:
      - 10.1.11.0/24
  PublicSubnet2Param:
    Description: Public Subnet 2
    Type: String
    Default: 10.1.12.0/24
    AllowedValues:
      - 10.1.12.0/24

  KeyName:
    Type: AWS::EC2::KeyPair::KeyName
    Description: Name of an existing EC2 KeyPair to enable SSH access to the ECS instances

  ECSAMI:
    Description: AMI ID
    Type: AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>
    Default: /aws/service/ecs/optimized-ami/amazon-linux/recommended/image_id

  ECSInstanceType:
    Description: AMI ID
    Type: String
    Default: t2.medium
    AllowedValues: 
      - t2.micro
      - t2.medium
      - m1.small
      - m1.large
    Description: Enter t2.micro, m1.small, or m1.large. Default is t2.micro.  

Resources:

  ECSCluster:
    Type: AWS::ECS::Cluster
    Properties:
      ClusterName: ECSCluster

  MyPrivateDnsNamespace:
    Type: "AWS::ServiceDiscovery::PrivateDnsNamespace"
    Properties:
      Vpc: !Ref VPC
      Name: MyPrivateDnsNamespace

  VPC:
    Type: 'AWS::EC2::VPC'
    Properties:
      CidrBlock: !Ref VPCCIDR
      EnableDnsSupport: 'true'
      EnableDnsHostnames: 'true'
      Tags:
        - Key: Name
          Value: ECS VPC

  InternetGateway:
    Type: 'AWS::EC2::InternetGateway'
    DependsOn: VPC
  AttachGateway:
    Type: 'AWS::EC2::VPCGatewayAttachment'
    DependsOn:
      - VPC
      - InternetGateway
    Properties:
      VpcId: !Ref VPC
      InternetGatewayId: !Ref InternetGateway

  PublicSubnet1:
    Type: 'AWS::EC2::Subnet'
    DependsOn: AttachGateway
    Properties:
      VpcId: !Ref VPC
      CidrBlock: !Ref PublicSubnet1Param
      AvailabilityZone: !Select
        - '0'
        - !GetAZs ''
      Tags:
        - Key: Name
          Value: Public Subnet 1

  PublicSubnet2:
    Type: 'AWS::EC2::Subnet'
    DependsOn: AttachGateway
    Properties:
      VpcId: !Ref VPC
      CidrBlock: !Ref PublicSubnet2Param
      AvailabilityZone: !Select
        - '1'
        - !GetAZs ''
      Tags:
        - Key: Name
          Value: Public Subnet 2

  PublicRouteTable:
    Type: 'AWS::EC2::RouteTable'
    DependsOn:
      - VPC
      - AttachGateway
    Properties:
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: Public

  PublicRoute:
    Type: 'AWS::EC2::Route'
    DependsOn:
      - PublicRouteTable
      - AttachGateway
    Properties:
      RouteTableId: !Ref PublicRouteTable
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId: !Ref InternetGateway

  PublicSubnet1RouteTableAssociation:
    Type: 'AWS::EC2::SubnetRouteTableAssociation'
    DependsOn:
      - PublicRouteTable
      - PublicSubnet1
      - AttachGateway
    Properties:
      SubnetId: !Ref PublicSubnet1
      RouteTableId: !Ref PublicRouteTable

  PublicSubnet2RouteTableAssociation:
    Type: 'AWS::EC2::SubnetRouteTableAssociation'
    DependsOn:
      - PublicRouteTable
      - PublicSubnet2
    Properties:
      SubnetId: !Ref PublicSubnet2
      RouteTableId: !Ref PublicRouteTable

  MySecurityGroup:
    Type: 'AWS::EC2::SecurityGroup'
    DependsOn: AttachGateway
    Properties:
      GroupDescription: MySecurityGroup
      GroupName: MySecurityGroup
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: MySecurityGroup

      SecurityGroupEgress:
        - IpProtocol: tcp
          FromPort: '0'
          ToPort: '65535'
          CidrIp: 0.0.0.0/0
        - IpProtocol: udp
          FromPort: '0'
          ToPort: '65535'
          CidrIp: 0.0.0.0/0
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: '80'
          ToPort: '80'
          CidrIp: 0.0.0.0/0

  MyAutoScallingGroup:
    DependsOn: EcsInstanceLaunchConfiguration
    Type: AWS::AutoScaling::AutoScalingGroup
    Properties:
      AutoScalingGroupName: MyAutoScallingGroup
      AvailabilityZones:
      - Fn::Select:
        - '0'
        - Fn::GetAZs:
            Ref: AWS::Region
      - Fn::Select:
        - '1'
        - Fn::GetAZs:
            Ref: AWS::Region
      LaunchConfigurationName:
        Ref: "EcsInstanceLaunchConfiguration"
      MinSize: '1'
      MaxSize: 1
      DesiredCapacity: 1
      VPCZoneIdentifier:
        - !Ref PublicSubnet1
        - !Ref PublicSubnet2

  myLoadBalancer:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    DependsOn:
      - FirstEIP
      - SecondEIP
    Properties:
      Type: network
      Name: MyLoadBalancer
      SubnetMappings:
        - AllocationId: !GetAtt
            - FirstEIP
            - AllocationId
          SubnetId: !Ref PublicSubnet1
        - AllocationId: !GetAtt
            - SecondEIP
            - AllocationId
          SubnetId: !Ref PublicSubnet2

  FirstEIP:
    Type: AWS::EC2::EIP
    Properties:
      Domain: VPC

  SecondEIP:
    Type: AWS::EC2::EIP
    Properties:
      Domain: VPC

  myListener:
    DependsOn:
      - myTargetGroup
      - myLoadBalancer
    Type: AWS::ElasticLoadBalancingV2::Listener
    Properties:
      DefaultActions:
      - Type: forward
        TargetGroupArn:
          Ref: myTargetGroup
      LoadBalancerArn:
        Ref: myLoadBalancer
      Port: '80'
      Protocol: TCP

  myTargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      Name: myTargetGroup
      Port: 80
      Protocol: TCP
      TargetType: instance
      VpcId: !Ref VPC


  ECSEC2RoleInstanceProfile:
    Type: AWS::IAM::InstanceProfile
    Properties:
      Path: "/"
      Roles:
      - !Ref ECSEC2Role

  ECSEC2Role:
    Type: AWS::IAM::Role
    Properties:
      RoleName: ECSEC2Role
      AssumeRolePolicyDocument:
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - ec2.amazonaws.com
          Action:
          - sts:AssumeRole
      Path: "/"
      Policies:
      - PolicyName: ecs-service
        PolicyDocument:
          Statement:
          - Effect: Allow
            Action:
            - ecs:*
            Resource: "*"

  EcsInstanceLaunchConfiguration:
    Type: AWS::AutoScaling::LaunchConfiguration
    Metadata:
      AWS::CloudFormation::Init:
        configSets:
          ConfigCluster:
            - Install
        Install:
          files:
            /home/ec2-user/.aws/config:
              mode: '000755'
              owner: ec2-user
              group: root
              content: !Sub |
                [default]
                region = ${AWS::Region}
            /etc/ecs/ecs.config:
              content: !Sub 'ECS_CLUSTER=${ECSCluster}'
              mode: '000755'
              owner: root
              group: root
    Properties:
      ImageId: !Ref ECSAMI
      LaunchConfigurationName: 'EcsInstanceLaunchConfiguration'
      InstanceType: !Ref ECSInstanceType
      AssociatePublicIpAddress: true
      IamInstanceProfile: !Ref 'ECSEC2RoleInstanceProfile'
      KeyName: !Ref 'KeyName'
      SecurityGroups:
        - !Ref 'MySecurityGroup'
      UserData:
        Fn::Base64:
          !Sub |
            #!/bin/bash -xe
            yum update -y
            yum install -y aws-cfn-bootstrap git aws-cli
            # Install the files and packages from the metadata
            /opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource EcsInstanceLaunchConfiguration --configsets ConfigCluster --region ${AWS::Region}
            # Signal the status from cfn-init
            /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackName} --resource MyAutoScallingGroup --region ${AWS::Region}

With params:

ECSAMI	/aws/service/ecs/optimized-ami/amazon-linux/recommended/image_id	ami-0b2cc421c0d3015b4
ECSInstanceType 	t2.medium
KeyName	einsteinish	
PublicSubnet1Param	10.1.11.0/24	
PublicSubnet2Param	10.1.12.0/24	
VPCCIDR	10.1.0.0/16

It will create the following resources:

From the stack run, we now have 2 instances from ECS-Optimized Amazon Linux 2 AMI:

And the ECS cluster:

If we need to create "ecsServiceRole", we may want to add the following to the template and update the stack:

  ecsServiceRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: ecsServiceRole
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - ecs.amazonaws.com
            Action:
              - sts:AssumeRole
      Path: /
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceRole

Create a Task Definition

Let's go to the ECS console and create a new "Task Definition". Select "EC2" launch type.

Click "Add volume":

We can see it's been added:

Now, we want to add two containers. Click "Add container" for "sample-app":

Click "Add container" for "busybox":

Note that we could have used the following json file to create the "Task Definition":

 {
  "family": "myECSApp",
  "containerDefinitions": [
    {
      "volumesFrom": [],
      "portMappings": [
        {
          "hostPort": 80,
          "containerPort": 80
        }
      ],
      "command": null,
      "environment": [],
      "essential": true,
      "entryPoint": null,
      "links": [],
      "mountPoints": [
        {
          "containerPath": "/usr/local/apache2/htdocs",
          "sourceVolume": "my-vol",
          "readOnly": null
        }
      ],
      "memory": 300,
      "name": "simple-app",
      "cpu": 10,
      "image": "httpd:2.4"
    },
    {
      "volumesFrom": [
        {
          "readOnly": null,
          "sourceContainer": "simple-app"
        }
      ],
      "portMappings": [],
      "command": [
       "/bin/sh -c \"while true; do echo '<html> <head> <title>Amazon ECS Sample App</title> <style>body {margin-top: 40px; background-color: #333;} </style> </head><body> <div style=color:white;text-align:center> <h1>Amazon ECS Sample App</h1> <h2>Congratulations!</h2> <p>Your application is now running on a container in Amazon ECS.</p>' > top; /bin/date > date ; echo '</div></body></html>' > bottom; cat top date bottom > /usr/local/apache2/htdocs/index.html ; sleep 1; done\""
      ],
      "environment": [],
      "essential": false,
      "entryPoint": [
        "sh",
        "-c"
      ],
      "links": [],
      "mountPoints": [],
      "memory": 200,
      "name": "busybox",
      "cpu": 10,
      "image": "busybox"
    }
  ],
  "volumes": [
    {
      "host": {
        "sourcePath": null
      },
      "name": "my-vol"
    }
  ]
}

Here is the Task Definition console screen before we click the "Create" button.

task-definition-1-config-task-and-container-definitions.png

Create an ECS Service

Time to create ECS Service for our containers. Go to ECS console and click "ECSCluster". Then, click "Create":

Click "Next step":

Finally, review the service and click "Create Service":

From the console, "Details" tab => "myTargetGroup" => "Targets" tab :

Paste the DNS into a browser:

Deploying a new application - Service update

After we update the Docker image, we need to create a new task definition with that image and deploy it to our service one at a time. The service scheduler create a task with the new task definition, and after it reaches "running" state, then the old task is drained and stopped. This process continues until all the tasks in our service are using the new task definition.

This time, we want to work with the following json file which is basically the same as the one before but we modified a line of the code:

 {
  "family": "myECSApp",
  "containerDefinitions": [
    {
      "volumesFrom": [],
      "portMappings": [
        {
          "hostPort": 80,
          "containerPort": 80
        }
      ],
      "command": null,
      "environment": [],
      "essential": true,
      "entryPoint": null,
      "links": [],
      "mountPoints": [
        {
          "containerPath": "/usr/local/apache2/htdocs",
          "sourceVolume": "my-vol",
          "readOnly": null
        }
      ],
      "memory": 300,
      "name": "simple-app",
      "cpu": 10,
      "image": "httpd:2.4"
    },
    {
      "volumesFrom": [
        {
          "readOnly": null,
          "sourceContainer": "simple-app"
        }
      ],
      "portMappings": [],
      "command": [
       "/bin/sh -c \"while true; do echo '<html> <head> <title>Amazon ECS Sample App</title> <style>body {margin-top: 40px; background-color: #333;} </style> </head><body> <div style=color:white;text-align:center> <h1>Amazon ECS Sample App</h1> <h2>Version 2.0.0 - Congratulations!</h2> <p>Your application is now running on a container in Amazon ECS.</p>' > top; /bin/date > date ; echo '</div></body></html>' > bottom; cat top date bottom > /usr/local/apache2/htdocs/index.html ; sleep 1; done\""
      ],
      "environment": [],
      "essential": false,
      "entryPoint": [
        "sh",
        "-c"
      ],
      "links": [],
      "mountPoints": [],
      "memory": 200,
      "name": "busybox",
      "cpu": 10,
      "image": "busybox"
    }
  ],
  "volumes": [
    {
      "host": {
        "sourcePath": null
      },
      "name": "my-vol"
    }
  ]
}

Click "Configure via JSON" and paste the code. Click "Save". Then, click "Create":

Click "Update" button:

Modify the version to the "latest":

Click "Next step" couple of times, and then "Update Service". Paaste the LB DNS into a browser:

Clean-up

To clean resources, we first delete "ECSCluster" and delete our stack. Some of the namescapes may still stay there, however, in that case, we may want to delete the "service" and then "namespace" associated with the service:

$ aws servicediscovery list-services --region us-east-1
$ aws servicediscovery delete-service --id srv-6mxl6hytuqg4ubyi

$ aws servicediscovery list-namespace --id ns-fjmcqxnsm6vpgvf2
$ aws servicediscovery delete-namespace --id ns-lnsvh43pytiokqhn

Updating a running ECS Service

Let's update the service so that it just runs one task from two.

On the ECS console, select the ECSCluster, and then click service "Update". Set "Number of tasks" to "1".