Docker image based on https://hub.docker.com/r/jrottenberg/ffmpeg/ Modified to use AWS Amazon Linux and other Python and command line tools for AWS capabilities
Pre-requisites:
- Local machine must have the AWS CLI configured with sufficient permissions to * ECS Container Repository (ECR)
- Docker command line tools
- Working knowledge of Docker, Python, FFMPEG
- Create two buckets - one for raw videos, one for frames
- Create a File Gateway on EC2: https://aws.amazon.com/premiumsupport/knowledge-center/file-gateway-ec2/
- Point the File Gateway to the S3 bucket for frames
- Upload a video to the video bucket
In this step, we are building a docker image from scratch. We build an image based on Amazon Linux, and build FFMPEG from code. We also install tools to use AWS APIs from Python (boto3)
- Install docker command line tools on your local machine
- Create an ECR Repository in the AWS Console
- Update the batch-job.py in this git repo with your File Gateway mount information where it says CHANGEME. Also update with the name of your video files.
- From command line, change directories to /docker in this repo
docker build -t ffmpeg-amazonlinux .docker tag ffmpeg-amazonlinux:latest 392583147479.dkr.ecr.us-east-1.amazonaws.com/ffmpeg-amazonlinux:latestaws ecr get-login --no-include-email --region us-east-1- Copy and paste the output of the above command
docker push <your ECR URL>/ffmpeg-amazonlinux:latest
- Configure AWS Batch in the AWS Console: https://docs.aws.amazon.com/batch/latest/userguide/Batch_GetStarted.html
- Ensure the Compute Environment is configured with connectivity to the File Gateway. See AWS VPC and Security Groups for more details.
- Create a job definition with your container image above
- The command should be similar to
python3 /batch/batch-job.py <your s3 bucket> - Set the job definition to priviledged and user to root: https://docs.aws.amazon.com/batch/latest/userguide/create-job-definition.html
- Submit the job to the job queue you created