Video Transcoding Service Project

This project demonstrates a video transcoding service using a combination of technologies including Terraform, TypeScript, Node.js, Redis, AWS, FFMPEG, and Docker. The service is designed to transcode videos into multiple resolutions and scales.

Technologies Used

• Terraform: Infrastructure as Code (IaC) tool used to provision and manage AWS resources.
• TypeScript
• Node.js
• Redis: An in-memory data structure store, used as a database, cache, and message broker.
• AWS: Amazon Web Services
• FFMPEG: A complete, cross-platform solution to record, convert, and stream audio and video.
• Docker

Project Overview

The project begins with a Lambda function that is attached to an AWS API Gateway. This function provides a presigned POST URL to upload videos to a source video bucket. Once a video file is uploaded to the source S3 bucket, it triggers an AWS Lambda function that adds the video processing task to an AWS SQS queue. An AWS ECS service running the transcoder polls this SQS queue for new tasks.

The transcoder Docker image takes the key of the video from the source video bucket and the destination video bucket. It uses FFMPEG to transcode the video into different scales for multiple resolutions. Once the transcoding is complete, the transcoded videos are uploaded to the destination S3 bucket.

To keep track of every key that is being processed, Redis is used to store key-value pairs of the video key and its current state (e.g., Processing, Complete, Failed).

Architecture

Key Points

• Scalability: The use of AWS and Docker ensures that the service can scale to handle large volumes of video transcoding requests.
• Efficiency: By using FFMPEG for video transcoding and Redis for state management, the service is efficient in processing and tracking video transcoding tasks.
• Reliability: The infrastructure is provisioned and managed using Terraform, ensuring that the service is reliable and consistent across deployments.

Getting Started

To get started with this project, you will need to:

1. Set up your AWS account and configure the necessary permissions for Terraform, Lambda, ECS, and S3.

2. Install Terraform, Node.js, Docker, and FFMPEG on your local machine.

3. Clone this repository and navigate to the project directory.

4. Run terraform init to initialize the Terraform configuration.

5. Make the necessary .tfvars file.

6. Run terraform apply to provision the AWS resources.

7. Deploy the Lambda function and ECS task definition using the provided scripts.

8. Building and Pushing the Transcoder Docker Image to Amazon ECR

   1. Navigate to the Amazon ECR page in your AWS Management Console.
   2. Obtain the push commands provided by ECR for your repository.
   3. Execute the push commands in your terminal to build the transcoder Docker image and push it to your ECR repository.

9. Test the service by uploading a video to the source S3 bucket and verifying that the transcoded videos are uploaded to the destination bucket.

Note: In some time, I will automate this process to streamline the deployment and setup of the transcoder Docker image.

Contributing

Contributions are welcome! Please feel free to submit pull requests or open issues to improve the project.