A step-by-step guide on creating an image thumbnail generator using AWS Lambda, S3 event notifications, and Terraform to automate serverless workflows.

Introduction

In the rapidly evolving digital landscape, efficient image processing is crucial for content creators aiming to maintain a compelling online presence. AWS Lambda image processing offers a scalable, serverless solution to automate tasks such as generating image thumbnails. This tutorial will guide you through building an image thumbnail generator using AWS Lambda, Amazon S3 event notifications, and Terraform. By the end, you’ll have a robust, automated workflow that seamlessly handles image processing tasks.

Prerequisites

Before diving into the implementation, ensure you have the following:

An AWS account with necessary permissions.
Terraform installed on your local machine.
Basic knowledge of AWS services, particularly S3 and Lambda.
Python installed for writing the Lambda function.

Step-by-Step Guide

1. Create Source and Destination Buckets

Start by setting up two Amazon S3 buckets: one to store the original images and another to save the generated thumbnails.

################################################################################
# S3 Source Image Bucket
################################################################################
resource "aws_s3_bucket" "source_image_bucket" {
  bucket = var.source_bucket_name
  tags = merge(
    local.common_tags,
    {
      Name = "${local.naming_prefix}-s3-source-bucket"
    }
  )
}

################################################################################
# S3 Thumbnail Image Bucket
################################################################################
resource "aws_s3_bucket" "thumbnail_image_bucket" {
  bucket = var.thumbnail_bucket_name
  tags = merge(
    local.common_tags,
    {
      Name = "${local.naming_prefix}-s3-thumbnail-bucket"
    }
  )
}

2. Create an IAM Policy

Define a policy that grants the Lambda function permissions to read from the source bucket and write to the destination bucket.

################################################################################
# S3 Policy to Get and Put objects
################################################################################
resource "aws_iam_policy" "lambda_s3_policy" {
  name   = "LambdaS3Policy"
  policy = jsonencode({
    "Version": "2012-10-17",
    "Statement": [
      {
        "Effect": "Allow",
        "Action": "s3:GetObject",
        "Resource": "${aws_s3_bucket.source_image_bucket.arn}/*"
      },
      {
        "Effect": "Allow",
        "Action": "s3:PutObject",
        "Resource": "${aws_s3_bucket.thumbnail_image_bucket.arn}/*"
      }
    ]
  })
}

3. Create a Lambda IAM Role

Attach the created policy along with the AWSLambdaBasicExecutionRole to a new IAM role.

################################################################################
# Lambda IAM role to assume the role
################################################################################
resource "aws_iam_role" "lambda_s3_role" {
  name = "LambdaS3Role"
  assume_role_policy = jsonencode({
    "Version": "2012-10-17",
    "Statement": [
      {
        "Effect": "Allow",
        "Principal": {
          "Service": "lambda.amazonaws.com"
        },
        "Action": "sts:AssumeRole"
      }
    ]
  })
}

################################################################################
# Assign policy to the role
################################################################################
resource "aws_iam_policy_attachment" "assigning_policy_to_role" {
  name       = "AssigningPolicyToRole"
  roles      = [aws_iam_role.lambda_s3_role.name]
  policy_arn = aws_iam_policy.lambda_s3_policy.arn
}

resource "aws_iam_policy_attachment" "assigning_lambda_execution_role" {
  name       = "AssigningLambdaExecutionRole"
  roles      = [aws_iam_role.lambda_s3_role.name]
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}

4. Create a Lambda Function

Develop the Python code for image processing, zip it, and then create a Lambda function using Python 3.12. Integrate Lambda Layers from Klayers for additional dependencies.

################################################################################
# Compressing lambda_handler function code
################################################################################
data "archive_file" "thumbnail_lambda_source_archive" {
  type        = "zip"
  source_dir  = "${path.module}/lambda"
  output_path = "${path.module}/lambda_function.zip"
}

################################################################################
# Creating Lambda Function
################################################################################
resource "aws_lambda_function" "create_thumbnail_lambda_function" {
  function_name    = "CreateThumbnailLambdaFunction"
  filename         = "${path.module}/lambda_function.zip"
  runtime          = "python3.12"
  handler          = "thumbnail_generator.lambda_handler"
  memory_size      = 256
  timeout          = 300
  role             = aws_iam_role.lambda_s3_role.arn
  source_code_hash = data.archive_file.thumbnail_lambda_source_archive.output_base64sha256

  environment {
    variables = {
      DEST_BUCKET = aws_s3_bucket.thumbnail_image_bucket.bucket
    }
  }

  layers = [
    "arn:aws:lambda:us-east-1:770693421928:layer:Klayers-p312-Pillow:2"
  ]
}

################################################################################
# Lambda Function Permission to have S3 as a Trigger for Lambda Function
################################################################################
resource "aws_lambda_permission" "thumbnail_allow_bucket" {
  statement_id  = "AllowExecutionFromS3Bucket"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.create_thumbnail_lambda_function.arn
  principal     = "s3.amazonaws.com"
  source_arn    = aws_s3_bucket.source_image_bucket.arn
}

5. Set Up S3 Event Notifications

Configure the source S3 bucket to trigger the Lambda function upon new image uploads.

################################################################################
# Creating S3 Notification for Lambda when Object is uploaded in the Source Bucket
################################################################################
resource "aws_s3_bucket_notification" "thumbnail_notification" {
  bucket = aws_s3_bucket.source_image_bucket.id

  lambda_function {
    lambda_function_arn = aws_lambda_function.create_thumbnail_lambda_function.arn
    events              = ["s3:ObjectCreated:*"]
  }

  depends_on = [
    aws_lambda_permission.thumbnail_allow_bucket
  ]
}

6. Create a CloudWatch Log Group

Set up a CloudWatch log group to capture logs from the Lambda function for monitoring and debugging.

################################################################################
# Creating CloudWatch Log group for Lambda Function
################################################################################
resource "aws_cloudwatch_log_group" "create_thumbnail_lambda_function_cloudwatch" {
  name              = "/aws/lambda/${aws_lambda_function.create_thumbnail_lambda_function.function_name}"
  retention_in_days = 30
}

7. Write Python Code for the Lambda Function

Develop the Lambda function using Python, leveraging the boto3 library for AWS interactions and Pillow for image processing.

import logging
import boto3
from io import BytesIO
from PIL import Image
import os

logger = logging.getLogger()
logger.setLevel(logging.INFO)

s3_client = boto3.client('s3')

def lambda_handler(event, context):
    logger.info(f"event: {event}")
    logger.info(f"context: {context}")

    # Get the S3 bucket and object key from the event
    bucket = event["Records"][0]["s3"]["bucket"]["name"]
    key = event["Records"][0]["s3"]["object"]["key"]

    # Define the destination bucket and thumbnail key
    thumbnail_bucket = os.environ['DEST_BUCKET']
    thumbnail_name, thumbnail_ext = os.path.splitext(key)
    thumbnail_key = f"{thumbnail_name}_thumbnail{thumbnail_ext}"
    logger.info(f"Bucket name: {bucket}, file name: {key}, Thumbnail Bucket name: {thumbnail_bucket}, file name: {thumbnail_key}")

    # Open the image using Pillow
    file_byte_string = s3_client.get_object(Bucket=bucket, Key=key)['Body'].read()
    img = Image.open(BytesIO(file_byte_string))
    logger.info(f"Size before compression: {img.size}")

    # Create a thumbnail
    img.thumbnail((500, 500))
    logger.info(f"Size after compression: {img.size}")

    # Save the thumbnail to a BytesIO object
    buffer = BytesIO()
    img.save(buffer, "JPEG")
    buffer.seek(0)

    # Upload the thumbnail to the destination bucket
    sent_data = s3_client.put_object(Bucket=thumbnail_bucket, Key=thumbnail_key, Body=buffer)
    if sent_data['ResponseMetadata']['HTTPStatusCode'] != 200:
        raise Exception(f"Failed to upload image {key} to bucket {bucket}")

    return event

8. Deploy with Terraform

Initialize Terraform, plan the deployment, and apply the configuration.

terraform init
terraform plan
terraform apply -auto-approve

Upon successful completion, Terraform will output relevant information confirming the resources have been created.

9. Testing

To ensure everything is set up correctly:

Upload an image to the source S3 bucket.
Verify that a thumbnail is generated in the destination bucket.
Check the CloudWatch log group for logs related to the Lambda function execution.

10. Cleanup

To avoid incurring unnecessary costs, delete the deployed resources when they’re no longer needed.

terraform destroy -auto-approve

Ensure you empty the S3 buckets before destroying them to comply with AWS requirements.

Conclusion

By following this guide, you’ve successfully built a serverless image thumbnail generator using AWS Lambda image processing, S3 event notifications, and Terraform. This architecture not only ensures scalability and reliability but also automates the workflow, allowing you to focus on creating quality content without worrying about manual image processing tasks.

For content creators seeking to streamline their workflow further, ThumblifyAI offers an advanced AI-powered thumbnail creation platform. Enhance your content’s visual appeal effortlessly and boost engagement with professional-quality thumbnails generated in seconds.

Start optimizing your thumbnail creation process today with ThumblifyAI!