Serverless ETL With Cloud Functions

Fivetran introduces a cloud-functions connector for syncing custom data sources into a data warehouse. The process involves writing a tiny function that fetches data from the custom source and hosting it on a serverless computing platform like Google Cloud Functions, AWS Lambda or Azure Functions. Fivetran calls this function every 15 minutes to fetch new data, deduplicates it, and merges it into the warehouse. The cloud-functions connector supports custom APIs, obscure APIs with few users, and data formats that are not self-describing like Protobuf.