Setup AWS ECS to deliver logs to SumoLogic with Launch Configuration settings

Setup AWS ECS to deliver logs to SumoLogic with Launch Configuration settings

Symptoms

When using AWS ECS with ECS optimized AMI to build the cluster and set 'sumologic' as the logging driver on containers, the task cannot be started correctly and got following events in Cluster - Service - Events:

Figure 1 - Events for no container instances met all requirements

Root Cause

The problem is the latest version of ECS optimized AMI does not include the binaries and settings for SumoLogic Docker Logging Driver in ECS agent.

Fix

• We need pre-install the plugin in AMI with

• We need to add extra script steps in "User Data" of "Launch Configuration" for including correct binaries and settings.

Step 1 - Prepare customized AMI with plugin installed

• Find latest ECS optimized AMI in your region and create a new instance
• SSH to the instance and install plugin with

docker plugin install store/sumologic/docker-logging-driver:1.0.2 --alias sumologic --grant-all-permissions

• Create AMI from the instance

Step 2 - Find "Launch Configuration" of current ECS cluster

Open the cluster console switch to "ECS Instances" tab, and click "Actions", then "View Cluster Resources":

Figure 2 - View cluster resources of ECS cluster

And in pop-up panel, click the link of "Launch configuration"

Step 3 - Create a copy of "Launch Configuration"

Since cannot edit "Launch configuration", we need to create a copy of it and apply following changes:

• Choose the modified AMI

Figure 2 - Copy Launch configuration with modified AMI

• Append script line into "User Data"

Figure 3 - Copy Launch configuration with extra script line in user data

Add following line into "User Data":

echo 'ECS_AVAILABLE_LOGGING_DRIVERS=["json-file","awslogs","sumologic"]' >> /etc/ecs/ecs.config

And then save it with a new name.

Step 4 - Associate "Auto Scaling Group" to new "Launch Configuration"

Go back to the "View Cluster Resources" in cluster console. And then click the link of "Auto Scaling group". Find the Auto Scaling group used by cluster and right click - "Edit". Then change the "Launch configuration" to the one we just created.

Figure 4 - Change Launch configuration in Auto Scaling group

Step 5 - Terminate all instances and wait new instances launched

Kill and drain all instances running with current configuration and wait auto scaling group launch them again. After instances re-launched, you can verify the service is running in "Events":

Figure 5 - Verify service is running