Orchestrating ELT in Airflow: Scheduling vs. Orchestrating

The text discusses the evaluation of an external scheduler for Fivetran data syncs and introduces Airflow as a tool for orchestrating workflows in the modern data stack. It explains that while Fivetran automates syncs between sources and destinations, minimizing latency to completely land a transformed dataset within a data warehouse requires intricate timing and hierarchy of dependencies. An external scheduler specifically designed for orchestrating workflows can meet these requirements. The text also differentiates between scheduling and orchestrating, highlighting that while scheduling is simple and sequential, orchestration occurs at the workflow-level and produces a dynamic, extendable, distributed, and robust DAG.