Anonymize Data with S3 Object Lambda

AWS recently introduced support for S3 Object Lambda, allowing users to anonymize data at access time using tools such as Gretel Transforms. This blog demonstrates how to launch a real-time S3 object transformation service using the following stack: AWS Lambda with containers, managed by Serverless; AWS API Gateway; S3; and Gretel Transforms. The solution design involves creating an API endpoint for generating pre-signed URLs to fetch and transform S3 objects. To test this solution, a pipeline is constructed using Gretel Transforms, which can be used to transform entire DataFrames. A Docker container is built on the AWS Lambda Python image, containing the handler.py module. The serverless.yml file manages the AWS infrastructure and resources needed for the system.