AWS ECS Deployment Tracking

Automatically track ECS deployments and extract container image digests using a Lambda function triggered by EventBridge.

Overview

This guide shows you how to deploy a Lambda function that:

Triggers automatically on ECS deployment completion events
Extracts container image SHA256 digests from task definitions
Posts deployment data to Cardinal for release correlation

How It Works

ECS Service Deployment Completes
    ↓
EventBridge triggers Lambda
    ↓
Lambda fetches task definition details
    ↓
Queries ECR for image SHA256 digests
    ↓
POSTs to Cardinal API

Prerequisites

AWS Account with appropriate permissions:
- cloudformation:*
- lambda:*
- iam:CreateRole, iam:AttachRolePolicy
- events:*
AWS CLI installed and configured
Cardinal API Key - Get this from your Cardinal account settings
At least one ECS cluster with a service running

Deployment Steps

1. Save the CloudFormation Template

Create a file named ecs-deployment-tracker.yaml with the following content:

AWSTemplateFormatVersion: '2010-09-09'
Description: 'ECS Deployment Tracker - Captures image SHAs from ECS deployments'
 
Parameters:
  CardinalApiKey:
    Type: String
    Description: 'CardinalHQ API key for authentication'
    NoEcho: true
 
  DeploymentEndpointUrl:
    Type: String
    Description: 'Cardinal API endpoint URL'
    Default: 'https://app.cardinalhq.io/_/chip/workloads'
 
Resources:
  LambdaExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub '${AWS::StackName}-ecs-tracker-role'
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: lambda.amazonaws.com
            Action: sts:AssumeRole
      Policies:
        - PolicyName: CloudWatchLogs
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action: logs:CreateLogGroup
                Resource: !Sub 'arn:aws:logs:${AWS::Region}:${AWS::AccountId}:*'
              - Effect: Allow
                Action:
                  - logs:CreateLogStream
                  - logs:PutLogEvents
                Resource: !Sub 'arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/lambda/${AWS::StackName}-ecs-deployment-tracker:*'
        - PolicyName: ECSReadAccess
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - ecs:DescribeServices
                  - ecs:DescribeTaskDefinition
                Resource: '*'
        - PolicyName: ECRReadAccess
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - ecr:DescribeImages
                  - ecr:BatchGetImage
                Resource: '*'
 
  DeploymentTrackerFunction:
    Type: AWS::Lambda::Function
    Properties:
      FunctionName: !Sub '${AWS::StackName}-ecs-deployment-tracker'
      Runtime: python3.13
      Handler: index.lambda_handler
      Role: !GetAtt LambdaExecutionRole.Arn
      Timeout: 60
      Environment:
        Variables:
          DEPLOYMENT_ENDPOINT_URL: !Ref DeploymentEndpointUrl
          CARDINALHQ_API_KEY: !Ref CardinalApiKey
      Code:
        ZipFile: |
          import boto3
          import json
          import logging
          import sys
          import os
          from urllib import request, error
          from typing import Dict, Optional
 
          ecs = boto3.client("ecs")
          ecr = boto3.client("ecr")
 
          logger = logging.getLogger()
          logger.setLevel(logging.INFO)
          handler = logging.StreamHandler(sys.stdout)
          handler.setFormatter(logging.Formatter('%(message)s'))
          logger.handlers = [handler]
 
          def log_json(**kwargs):
              logger.info(json.dumps(kwargs))
 
          def get_image_digest(image_uri: str) -> Optional[str]:
              try:
                  if '@sha256:' in image_uri:
                      return image_uri.split('@')[1]
 
                  parts = image_uri.split('/')
                  if len(parts) < 2:
                      return None
 
                  repo_and_tag = '/'.join(parts[1:])
                  if ':' in repo_and_tag:
                      repository, tag = repo_and_tag.rsplit(':', 1)
                  else:
                      repository = repo_and_tag
                      tag = 'latest'
 
                  response = ecr.describe_images(
                      repositoryName=repository,
                      imageIds=[{'imageTag': tag}]
                  )
 
                  if response['imageDetails']:
                      digest = response['imageDetails'][0]['imageDigest']
                      log_json(level="debug", msg="Found digest", image=image_uri, digest=digest)
                      return digest
 
                  return None
 
              except ecr.exceptions.ImageNotFoundException:
                  log_json(level="warn", msg="Image not found in ECR", image=image_uri)
                  return None
              except Exception as e:
                  log_json(level="error", msg="Failed to get image digest", image=image_uri, error=str(e))
                  return None
 
          def post_deployment(payload: Dict) -> bool:
              endpoint = os.environ.get('DEPLOYMENT_ENDPOINT_URL', 'https://app.cardinalhq.io/_/chip/workloads')
              api_key = os.environ.get('CARDINALHQ_API_KEY')
 
              if not api_key:
                  log_json(level="error", msg="CARDINALHQ_API_KEY not set")
                  return False
 
              try:
                  headers = {
                      'Content-Type': 'application/json',
                      'x-cardinalhq-api-key': api_key,
                  }
 
                  data = json.dumps(payload).encode('utf-8')
                  req = request.Request(endpoint, data=data, headers=headers, method='POST')
 
                  log_json(level="info", msg="Posting to endpoint", endpoint=endpoint, payload=payload)
 
                  with request.urlopen(req, timeout=10) as response:
                      status = response.getcode()
                      body = response.read().decode('utf-8')
                      log_json(level="info", msg="Posted to endpoint", status=status, response=body)
                      return status >= 200 and status < 300
 
              except error.HTTPError as e:
                  log_json(level="error", msg="HTTP error posting deployment",
                          status=e.code, error=e.read().decode('utf-8'))
                  return False
              except Exception as e:
                  log_json(level="error", msg="Failed to post deployment", error=str(e))
                  return False
 
          def lambda_handler(event, context):
              log_json(event="raw_ecs_event", data=event)
 
              resources = event.get("resources", [])
              if not resources:
                  log_json(level="warn", msg="No resources found in event")
                  return {"statusCode": 400, "body": "No resources"}
 
              service_arn = resources[0]
              parts = service_arn.split(":service/")[-1].split("/")
              if len(parts) != 2:
                  log_json(level="error", msg="Unexpected ARN format", arn=service_arn)
                  return {"statusCode": 400, "body": "Invalid ARN"}
 
              cluster_name, service_name = parts
 
              try:
                  svc_resp = ecs.describe_services(cluster=cluster_name, services=[service_name])
              except Exception as e:
                  log_json(level="error", msg="Failed to describe service", error=str(e))
                  return {"statusCode": 500, "body": "ECS API error"}
 
              if not svc_resp.get("services"):
                  log_json(level="error", msg="Service not found", cluster=cluster_name, service=service_name)
                  return {"statusCode": 404, "body": "Service not found"}
 
              service = svc_resp["services"][0]
              task_def_arn = service["taskDefinition"]
 
              try:
                  td_resp = ecs.describe_task_definition(taskDefinition=task_def_arn)
              except Exception as e:
                  log_json(level="error", msg="Failed to describe task definition", error=str(e))
                  return {"statusCode": 500, "body": "ECS API error"}
 
              task_def = td_resp["taskDefinition"]
              container_defs = task_def["containerDefinitions"]
 
              digests = []
              images_info = []
 
              for container in container_defs:
                  image_uri = container["image"]
                  digest = get_image_digest(image_uri)
 
                  if digest:
                      digests.append(digest)
 
                  images_info.append({
                      "containerName": container["name"],
                      "image": image_uri,
                      "digest": digest
                  })
 
              region = event.get("region", "us-east-1")
              scope = f"{region}/{cluster_name}"
 
              deployment_payload = {
                  "runtime": "ecs",
                  "scope": scope,
                  "workloads": [
                      {
                          "name": service_name,
                          "properties": {
                              "serviceArn": service_arn,
                              "cluster": cluster_name,
                              "region": region,
                              "deploymentId": event["detail"].get("deploymentId"),
                              "eventType": event["detail"].get("eventName"),
                              "taskDefinition": task_def_arn,
                              "images": images_info,
                              "timestamp": event["time"]
                          },
                          "digests": digests
                      }
                  ]
              }
 
              log_json(event="ecs_deployment_enriched", payload=deployment_payload)
              success = post_deployment(deployment_payload)
 
              return {
                  "statusCode": 200 if success else 500,
                  "body": json.dumps(deployment_payload)
              }
 
  DeploymentEventRule:
    Type: AWS::Events::Rule
    Properties:
      Name: !Sub '${AWS::StackName}-ecs-tracker-trigger'
      Description: 'Trigger Lambda on ECS deployment completion'
      State: ENABLED
      EventPattern:
        source:
          - aws.ecs
        detail-type:
          - ECS Deployment State Change
        detail:
          eventType:
            - SERVICE_DEPLOYMENT_COMPLETED
      Targets:
        - Arn: !GetAtt DeploymentTrackerFunction.Arn
          Id: DeploymentTrackerTarget
 
  LambdaInvokePermission:
    Type: AWS::Lambda::Permission
    Properties:
      FunctionName: !Ref DeploymentTrackerFunction
      Action: lambda:InvokeFunction
      Principal: events.amazonaws.com
      SourceArn: !GetAtt DeploymentEventRule.Arn
 
Outputs:
  LambdaFunctionArn:
    Description: 'ARN of the deployment tracker Lambda function'
    Value: !GetAtt DeploymentTrackerFunction.Arn
    Export:
      Name: !Sub '${AWS::StackName}-LambdaArn'
 
  LambdaFunctionName:
    Description: 'Name of the Lambda function'
    Value: !Ref DeploymentTrackerFunction
    Export:
      Name: !Sub '${AWS::StackName}-LambdaName'
 
  EventBridgeRuleArn:
    Description: 'ARN of the EventBridge rule'
    Value: !GetAtt DeploymentEventRule.Arn
 
  IAMRoleArn:
    Description: 'ARN of the Lambda execution role'
    Value: !GetAtt LambdaExecutionRole.Arn

2. Deploy the Stack

aws cloudformation create-stack \
  --stack-name ecs-deployment-tracker \
  --template-body file://ecs-deployment-tracker.yaml \
  --parameters ParameterKey=CardinalApiKey,ParameterValue=YOUR_API_KEY_HERE \
  --capabilities CAPABILITY_NAMED_IAM \
  --region us-east-1

Important: Replace YOUR_API_KEY_HERE with your actual Cardinal API key and adjust the region as needed.

Wait for the stack to complete:

aws cloudformation wait stack-create-complete \
  --stack-name ecs-deployment-tracker \
  --region us-east-1

3. Verify Deployment

Check the stack outputs:

aws cloudformation describe-stacks \
  --stack-name ecs-deployment-tracker \
  --region us-east-1 \
  --query 'Stacks[0].Outputs'

4. Test the Integration

Manual Test:

Create a test event file with your actual ECS service ARN:

cat > test-event.json << 'EOF'
{
  "resources": ["arn:aws:ecs:us-east-1:123456789012:service/my-cluster/my-service"],
  "detail": {
    "eventType": "SERVICE_DEPLOYMENT_COMPLETED",
    "deploymentId": "ecs-svc/1234567890"
  },
  "region": "us-east-1",
  "time": "2025-01-11T12:00:00Z"
}
EOF

Important: Edit the file and replace:

us-east-1 with your region
123456789012 with your AWS account ID
my-cluster with your cluster name
my-service with your service name

Invoke the Lambda:

aws lambda invoke \
  --function-name ecs-deployment-tracker-ecs-deployment-tracker \
  --payload file://test-event.json \
  --cli-binary-format raw-in-base64-out \
  --region us-east-1 \
  response.json
 
# Check the response
cat response.json

Check the logs:

aws logs tail /aws/lambda/ecs-deployment-tracker-ecs-deployment-tracker \
  --region us-east-1 \
  --since 5m

End-to-End Test:

Trigger a real deployment:

aws ecs update-service \
  --cluster my-cluster \
  --service my-service \
  --force-new-deployment \
  --region us-east-1

Wait for the deployment to complete, then check logs again to verify the Lambda was triggered automatically.

Payload Structure

The Lambda sends this payload to Cardinal:

{
  "runtime": "ecs",
  "scope": "us-east-1/prod-cluster",
  "workloads": [{
    "name": "my-service",
    "digests": [
      "sha256:4ccf70c2320f8c1131a4c56781aefc4f1a06fcd50bd4a87c63f3325d5fe09985"
    ],
    "properties": {
      "serviceArn": "arn:aws:ecs:us-east-1:123456789012:service/prod-cluster/my-service",
      "cluster": "prod-cluster",
      "region": "us-east-1",
      "deploymentId": "ecs-svc/1234567890123456789",
      "eventType": "SERVICE_DEPLOYMENT_COMPLETED",
      "taskDefinition": "arn:aws:ecs:us-east-1:123456789012:task-definition/my-service:42",
      "images": [{
        "containerName": "app",
        "image": "123456789012.dkr.ecr.us-east-1.amazonaws.com/my-app:v1.2.3",
        "digest": "sha256:4ccf70c2..."
      }],
      "timestamp": "2024-01-15T10:30:45.123Z"
    }
  }]
}

Key Fields:

runtime: Always "ecs"
scope: Format is {region}/{cluster-name}
digests: Array of SHA256 digests from all containers
properties: ECS-specific metadata

Resources Created

Resource	Purpose
Lambda Function	Processes ECS deployment events
IAM Role	Lambda execution role with minimal permissions
EventBridge Rule	Triggers Lambda on deployment completion
CloudWatch Log Group	Stores Lambda logs

Troubleshooting

Lambda not receiving events

Check EventBridge rule is enabled:

aws events describe-rule \
  --name ecs-deployment-tracker-ecs-tracker-trigger \
  --region us-east-1

Verify the Lambda has permission from EventBridge:

aws lambda get-policy \
  --function-name ecs-deployment-tracker-ecs-deployment-tracker \
  --region us-east-1

Lambda fails with permission errors

Verify the IAM role has required policies:

aws iam list-attached-role-policies \
  --role-name ecs-deployment-tracker-ecs-tracker-role

Image digest not found

If logs show "Image not found in ECR":

Verify the image exists in ECR
Check the Lambda role has ecr:DescribeImages permission
Ensure the repository name is correct

For external images (e.g., Docker Hub), digests cannot be retrieved from ECR. The Lambda will log a warning and continue without the digest.

HTTP error posting to Cardinal

Check environment variables are set:

aws lambda get-function-configuration \
  --function-name ecs-deployment-tracker-ecs-deployment-tracker \
  --region us-east-1 \
  --query 'Environment.Variables'

Verify:

CARDINALHQ_API_KEY is set correctly
DEPLOYMENT_ENDPOINT_URL is https://app.cardinalhq.io/_/chip/workloads

Check Lambda logs for the exact HTTP error:

aws logs filter-log-events \
  --log-group-name /aws/lambda/ecs-deployment-tracker-ecs-deployment-tracker \
  --filter-pattern "HTTP error" \
  --region us-east-1

Cleanup

To remove all resources:

aws cloudformation delete-stack \
  --stack-name ecs-deployment-tracker \
  --region us-east-1

Next Steps

After setting up ECS tracking:

Configure the Release Agent in Cardinal
Set up GitHub integration to correlate deployments with releases
Start asking your agent questions about deployments

CloudRun Terraform Config ECS Lambda Code