Astronomer

Founded in 2018. Privately Held.

External links: homepage | docs | blog | jobs | youtube | twitter | github | linkedin

Airflow data pipelines.

Blog posts published by month since the start of

191 total blog posts published.

Switch to word count

Blog content

post title author published words HN
Reliable Data Orchestration for AI Applications Devin Stein Feb. 28, 2024 902 -
3 Key Takeaways from Airflow Summit 2023 Viraj Parekh Sep. 29, 2023 603 -
3 Ways to Extract Data Lineage with Airflow Howard Yoo Nov. 11, 2022 2118 -
5 Ways to View and Manage DAGs in Airflow Craig Hubert Dec. 07, 2022 673 -
6 Lessons Learned in Building Astronomer’s Developer Documentation Jake Witz Jul. 25, 2023 1611 -
10 Best Practices for Airflow Users Julia Wrzosińska Feb. 08, 2022 2034 -
10 Best Practices for Modern Data Orchestration with Airflow Steve Swoyer Apr. 14, 2022 3576 -
Exploring Airflow 2.9 Features with Astronomer's 29 Days of Airflow 2.9 Series Radhika Gulati Jun. 20, 2024 953 -
A Short History of DAG Writing Pete DeJoy Nov. 28, 2022 1726 -
Accelerating ML Application Development: Production-Ready Airflow Integrations with Critical AI Tools Steven Hillion Nov. 28, 2023 995 -
Adding Data Quality to DAGs ft. Great Expectations Benji Lampel Jan. 03, 2022 1958 -
Advanced XCom Configurations and Trigger Rules Tips and Tricks to Level-Up your Airflow DAGs George Yates Dec. 08, 2023 1898 -
Advantages of Hosted Airflow for Your ETL Workflows George Yates Jul. 18, 2023 752 -
Airflow 2.4 and Data-Driven Scheduling: How a New Feature Is Saving Time at Astronomer Steve Swoyer Sep. 29, 2022 1419 -
The Airflow 2.0 Scheduler Vikram Koka Dec. 17, 2020 2399 -
Airflow and dbt, Hand in Hand Josh Fell Apr. 05, 2022 1264 -
Democratizing the Data Stack—Airflow for Business Workflows Viraj Parekh Oct. 28, 2021 863 -
Building a Scalable Analytics Architecture With Airflow and dbt Pete DeJoy Dec. 22, 2020 2302 -
Building a Scalable Analytics Architecture With Airflow and dbt: Part 2 Pete DeJoy Jan. 05, 2021 2268 -
Building a Scalable Analytics Architecture With Airflow and dbt: Part 3 Sam Bail Aug. 06, 2021 1377 -
A Great Expectations Provider for Apache Airflow® Pete DeJoy Nov. 23, 2020 1174 -
Everything You Need to Know About the Airflow Summit 2021 Joanna Kasprzak-Kajder May. 11, 2021 814 -
Airflow Summit 2022 — Join the Airflow Event of the Year! Bolke de Bruin Mar. 09, 2022 245 -
Apache Airflow® vs. Apache Beam Julia Wrzosińska Nov. 17, 2021 2054 -
Airflow Summit 2021 Highlights Joanna Kasprzak-Kajder Jul. 27, 2021 593 -
Announcing Astro’s HIPAA and PCI-DSS Compliance Craig Hubert Sep. 09, 2022 297 -
Announcing "The Data Flowcast" by Astronomer: Your Gateway to Mastering Airflow Briana Okyere May. 30, 2024 523 -
Apache Airflow® 2.3 — Everything You Need to Know Steve Swoyer May. 02, 2022 2182 -
Apache Airflow® 2.4 — Everything You Need to Know Steve Swoyer Sep. 19, 2022 2688 -
Everything You Need to Know About Apache Airflow® 2.2.0 Julia Wrzosińska Oct. 12, 2021 948 -
Apache Airflow® at Astronomer—Taking Data Orchestration to the Next Level Ula Rydiger Feb. 22, 2022 1424 -
Apache Airflow® for Data Leaders — How to Empower Data Teams Steven Hillion Mar. 10, 2022 1460 -
Apache Airflow® for Data Engineers—How to Leverage Data Orchestration Ula Rydiger Dec. 17, 2021 1347 -
Apache Airflow® for Data Scientists Julia Wrzosińska Apr. 21, 2022 2426 -
Apache Airflow® TaskFlow API vs. Traditional Operators: An In-Depth Comparison for Efficient DAGs George Yates Nov. 02, 2023 1727 -
Airflow at Wise: Data Orchestrator in Machine Learning Joanna Kasprzak-Kajder Sep. 16, 2021 1080 -
Apache NiFi vs. Apache Airflow® Julia Wrzosińska Sep. 22, 2021 1410 -
Improving Ask Astro: The Journey to Enhanced Retrieval Augmented Generation (RAG) with Cohere Rerank, Part 4 David Xue Mar. 07, 2024 1081 -
Ask Astro: An open source LLM Application with Apache Airflow®, Part 2 Michael Gregory Sep. 22, 2023 580 -
Ask Astro: Operationalizing Data Ingest for Retrieval Augmented Generation with LLMs, Part 3 Michael Gregory Oct. 24, 2023 2448 -
Astro by Astronomer Delivered 438% ROI: Insights from a Forrester TEI Study Astronomer Jan. 31, 2024 423 -
Astro CLI: The Easiest Way to Install Apache Airflow® Paola Peraza Calderon Sep. 22, 2022 1940 -
Introducing Astro, the Fully Managed Data Orchestration Platform, Powered by Airflow Joe Otto Jun. 10, 2022 1115 -
Incident Management at Astronomer: 1 Year Later Kevin Paulisse Feb. 28, 2024 2431 -
Astro Is Now Available on All Major Cloud Providers Ryan Fox-Tyler Aug. 09, 2022 402 -
What’s new in the Astro Platform Release, Q1 2024 Juliana O'Donohue Mar. 26, 2024 1195 -
Reimagining Airflow for Data Engineers and Data Scientists with the Astro Python SDK Kenten Danas Aug. 22, 2022 1121 -
Astronomer Acquires Datakin, the Data Lineage Tool Joe Otto Mar. 23, 2022 575 -
Astronomer Adopts DAG Factory to Democratize Writing Data Pipelines Viraj Parekh Jul. 17, 2024 492 -
Astronomer and Snowflake: Unleash the Power of Snowpark Container Services and Apache Airflow® Juliana O'Donohue Jun. 26, 2023 604 -
Astronomer and Fivetran Partner to Release Production-Grade ELT Airflow Provider Viraj Parekh Jul. 12, 2023 1364 -
Announcing the Astronomer Registry Pete DeJoy Mar. 30, 2021 882 -
Astronomer and Uturn Partner to Drive Innovation and Better Business Outcomes Will Murphy Dec. 22, 2021 446 -
Airflow at BBC—Data Orchestration Solution in Media Ula Rydiger Oct. 14, 2021 855 -
How to Select the Best ETL Tool to Integrate With Airflow? Our 3 Picks Julia Wrzosińska Dec. 03, 2021 2465 -
Best Practices for Building an Airflow Service (Part 1) Viraj Parekh Jul. 13, 2023 1764 -
Best Practices for Secure Network Connectivity and Authentication in Astro Deepak Narain Nov. 17, 2022 848 -
Big Data Architecture: Core Components, Use Cases, and Limitations Julia Wrzosińska Oct. 05, 2021 1824 -
How to Build a Modern Data Stack Julia Wrzosińska Nov. 19, 2021 1634 -
How to Build an ETL Process? Julia Wrzosińska Sep. 03, 2021 1495 -
To Build or to Buy? DIY Orchestration with Airflow vs. A Fully Managed Service Steve Swoyer May. 24, 2022 3726 -
Change Data Capture in Apache Airflow® - Part 1 Manmeet Kaur Rangoola Aug. 22, 2023 1540 -
Change Data Capture with Airflow - Part 2 Manmeet Kaur Rangoola Sep. 08, 2023 2225 -
Change Data Capture With Apache Airflow®: Part 1 Rob Deeb Jan. 21, 2021 5264 -
CI/CD Enforcement for Code Changes David Koenitzer Mar. 30, 2023 478 -
The Astro Cloud IDE: from Python and SQL to nearly 1,000 Airflow operators Julian LaNeve Jun. 13, 2023 1113 -
Comparing Data Orchestration: Databricks Workflows vs. Apache Airflow®, Part 1 George Yates Aug. 30, 2023 3691 -
Airflow at Societe Generale: Data Orchestration Solution in Banking Ula Rydiger Aug. 17, 2021 982 -
Data Pipeline: Components, Types, and Best Practices Julia Wrzosińska Aug. 09, 2021 1976 -
How Data Pipelines Drive Improved Sales in E-commerce Ula Rydiger Jul. 22, 2021 1193 -
Data Silos: What Are They and How to Fix Them? Ula Rydiger Aug. 24, 2021 1782 -
Databricks vs. Airflow From a Management Perspective, Part 2 George Yates Oct. 11, 2023 1859 -
Day 2 Operations for LLMs with Apache Airflow®: Going Beyond the Prototype, Part 1 Michael Gregory Sep. 21, 2023 1813 -
Debugging Airflow Made Easy: 3 Key Steps to Debug your Airflow DAGs Kenten Danas Aug. 24, 2023 1150 -
Deploy Rollbacks: Upgrade Airflow and Deploy DAGs with Confidence David Koenitzer Dec. 14, 2023 804 -
Data Orchestration: The Dividing Line Between Generative AI Success and Failure Steven Hillion May. 29, 2024 984 -
Enhanced Authentication Security to your Data Services on Azure with Astro Piotr Chomiak Dec. 01, 2023 691 -
ETL in Airflow: A Comprehensive Guide to Efficient Data Pipelines George Yates Aug. 01, 2023 1318 -
Every Company Nowadays Becomes a Data Company—Interview with Bolke de Bruin Bolke de Bruin Nov. 30, 2021 1416 -
Everything You Should Know About Airflow 2.3’s New Grid View Brent Bovenzi Aug. 05, 2022 1159 -
Expanding Data Access and Exchange Inside a Company Craig Hubert Oct. 04, 2022 1598 -
Expert tips for monitoring the health and SLAs of your Apache Airflow® DAGs Manmeet Kaur Rangoola Jul. 20, 2023 1880 -
The Future of Banking: How Can Apache Airflow® Help? Ula Rydiger Sep. 29, 2021 942 -
Validate Your Apache Airflow® Skills With the Astronomer Certification Marc Lamberti May. 05, 2021 760 -
How an Improved DAG-Testing Command in the Astro CLI Made Its Way into Airflow Steve Swoyer Dec. 09, 2022 1319 -
How Astro runs billions of Airflow tasks around the world Mehul Goyal Feb. 06, 2024 1443 2
How Astro’s Data Graph Helps Data Engineers Run and Fix Their Pipelines Conor Beverland Nov. 10, 2022 600 -
How to Keep Data Quality in Check with Airflow Tamara Fingerlin Oct. 27, 2022 3282 -
How we optimized the Registry for performance across millions of page views Julian LaNeve May. 15, 2023 812 2
How We Track the Growth of Apache Airflow® Ross Turk Sep. 01, 2022 1109 -
Get Improved Data Quality Checks in Airflow with the Updated Great Expectations Operator Benji Lampel Jan. 27, 2023 1636 -
Inside Authorized Workspaces, A New Feature in Astro Piotr Chomiak Feb. 24, 2023 701 -
Introducing Airflow 2.0 Paola Peraza Calderon Oct. 29, 2020 2990 2
Introducing Airflow 2.6 Viraj Parekh May. 02, 2023 1030 -
Introducing Airflow 2.7 Kenten Danas Aug. 18, 2023 1616 -
Introducing Airflow 2.8 Kenten Danas Dec. 18, 2023 1283 -
Introducing Apache Airflow® 2.9 Kenten Danas Apr. 08, 2024 1602 -
Introducing Apache Airflow® on Astro – an Azure Native ISV Service Conor Beverland Nov. 15, 2023 1735 -
Introducing Astronomer Providers Viraj Parekh May. 12, 2022 444 -
Introducing Astro’s New Workspace Homepage Juliana O'Donohue Jan. 19, 2023 343 -
Introducing Cohort 2 of the Astronomer Champions Program for Apache Airflow®! Briana Okyere Mar. 27, 2024 2413 -
Introducing Cohort 3 of the Astronomer Champions Program for Apache Airflow®! Briana Okyere Jun. 25, 2024 2445 -
Introducing Cosmos 1.0: the best way to run dbt Core in Airflow Julian LaNeve Jul. 26, 2023 1861 2
Introducing the First Generative AI Cookbook for Data Orchestration Kenten Danas Jun. 04, 2024 797 -
Kubernetes Executor Support in Astro: A New Era of Scalability and Resource Management Constance Martineau Apr. 03, 2023 459 -
Introducing SnowPatrol - Snowflake Anomaly Detection and Cost Management with Machine Learning and Airflow Olivier Daneau Mar. 19, 2024 2643 -
Introducing Support for the Kubernetes Executor on Astro, Now in Private Preview Juliana O'Donohue Feb. 23, 2023 193 -
Introducing the Astro Cloud IDE Julian LaNeve Dec. 05, 2022 1678 8
Introducing the Astro Platform Release, Q4 2023 Juliana O'Donohue Dec. 06, 2023 1275 -
Introducing the Astronomer Champions Program for Apache Airflow® Briana Okyere Jan. 16, 2024 1288 -
Enhanced Astro Workspace Roles for more granular permissions Piotr Chomiak Aug. 09, 2023 982 -
Letter from the CEO: Our Story So Far Joe Otto Mar. 23, 2022 1338 -
Leveraging Apache Airflow® and Kubernetes for Data Processing Constance Martineau Aug. 15, 2023 1038 -
Machine Learning Pipeline Orchestration Santona Tuli Nov. 23, 2021 3814 -
Machine Learning Pipeline: Everything You Need to Know Ula Rydiger Oct. 26, 2021 2566 -
Maximizing Data Workflow Efficiency: The Advantages of Using Airflow with Azure Data Factory George Yates Feb. 15, 2024 960 -
Micropipelines: A Microservice Approach for DAG Authoring in Apache Airflow® Vikram Koka Oct. 11, 2022 2814 -
Migrate Python Jobs to Airflow in 4 Simple Steps Manmeet Kaur Rangoola Oct. 06, 2023 2181 2
ML for Customer Analytics with Airflow, Snowpark, and Weaviate George Yates Oct. 10, 2023 4265 -
Introducing New Astro CLI Commands to Make DAG Testing Easier David Koenitzer Jun. 02, 2022 1101 -
OpenLineage Is on the Rise in 2023 Ross Turk Mar. 10, 2023 967 -
OpenLineage: Where It Came from and What Comes Next Craig Hubert Nov. 03, 2022 1474 -
How to orchestrate Azure Data Explorer queries with Airflow Kenten Danas Dec. 10, 2021 629 -
How to orchestrate Talend jobs with Airflow Kenten Danas Apr. 17, 2021 1668 -
Orchestrating Feature Pipelines: Announcing the Tecton Airflow Provider Nick Acosta Oct. 31, 2023 1174 -
Orchestrating Machine Learning Pipelines with Airflow George Yates Aug. 23, 2023 761 -
Orchestration, or How to Become a Data-Driven Company Steven Hillion Oct. 17, 2022 4768 -
Podcast Spotlight: What Observability Brings to Data Orchestration Steve Swoyer Sep. 14, 2022 845 -
What is Reverse ETL and How Can It Improve Data Flow? Sarah Krasnik Oct. 19, 2021 1690 -
Revolutionizing Data Orchestration and MLOps with Apache Airflow® and Astronomer at Chiper Johan Stiven Ruiz Arias Jan. 19, 2024 1350 -
Run ETL with Astro and CrateDB Cloud in 30min - fully up in the cloud Marija Selakovic Jul. 27, 2023 2237 -
Scale Airflow with confidence using Astro’s new Alerting capabilities Julian LaNeve May. 03, 2023 521 -
How to Manage Secrets in Apache Airflow® 2.0 Eric Griffing Mar. 08, 2021 550 -
SnowPatrol Series: Convert anomalies into actions, and how Grindr saved $600,000 in Snowflake costs Olivier Daneau May. 30, 2024 3030 -
Standardizing your Astro projects with Cookiecutter and Cruft Bas Harenslak Jan. 23, 2024 1962 -
Airflow 2.0 TaskFlow API and Its Features Eric Griffing Mar. 23, 2021 252 -
TechCrunch on Astronomer’s Big News Tom de Kay Mar. 23, 2022 117 -
Test Airflow Upgrades with the Astro CLI David Koenitzer Aug. 10, 2023 881 -
Ensuring Data Pipeline Integrity: A Comprehensive Guide to Testing Airflow DAGs Manmeet Kaur Rangoola Apr. 02, 2024 2008 -
The Airflow Year in Review 2022 Viraj Parekh Dec. 21, 2022 1364 -
The Astronomer Providers Package — A Better Option for Long-Running Tasks Steve Swoyer Jul. 14, 2022 996 -
Introducing KEDA for Airflow Daniel Imberman Mar. 17, 2020 1196 -
The New, Faster Way to Deploy Airflow DAGs to Astro Paola Peraza Calderon Dec. 12, 2022 994 -
The New KubernetesExecutor Daniel Imberman Apr. 21, 2021 1410 -
The Top 7 Alternatives to MWAA Astronomer Jun. 02, 2023 1488 -
The Top 7 Alternatives to Google Cloud Composer Astronomer Jun. 02, 2023 1474 -
Three ways to use Airflow with MotherDuck and DuckDB Tamara Fingerlin Jun. 22, 2023 1452 -
Top Data Management Trends for 2022 Ula Rydiger Jan. 06, 2022 2490 -
Tracking Innovation: How Astronomer Streamlined Release Notes with Towncrier Jake Witz Feb. 22, 2024 1226 -
Updates coming to cross-account role and permissions boundary for Enhanced Data Plane cluster management with Karpenter Astronomer Jul. 08, 2023 319 -
Using Astronomer’s new Cosmos to deploy dbt pipelines onto Snowflake George Yates Oct. 12, 2023 1615 -
Ventana Research Names Astronomer as a Finalist for Its Digital Innovation Awards Craig Hubert Jun. 17, 2022 170 -
Welcome to our new New York City headquarters! Astronomer Feb. 13, 2024 442 -
What Is Data Lineage and Why Does It Matter? Ross Turk Mar. 30, 2022 1520 -
What Is Data Orchestration and Why Is It Essential for Business Julia Wrzosińska Jul. 29, 2021 2378 -
What’s New in Apache Airflow® 2.5 Steve Swoyer Dec. 02, 2022 913 -
What’s New in Astro Python SDK 1.1: Data-Driven Scheduling, Dynamic Tasks, and Redshift Support Mike Shwe Oct. 18, 2022 1621 -
Win a Scholarship for CoRise’s “Effective Data Orchestration with Airflow” Course Astronomer Dec. 21, 2022 148 -
Airflow and dbt: the next chapter Julian LaNeve Jul. 31, 2024 1171 -
What’s New in the Astro Platform Release Radhika Gulati Jul. 31, 2024 1000 -
A Step-by-Step Guide to Automating Your Astro Infrastructure with the Astro Terraform Provider Bas Harenslak Aug. 02, 2024 1227 -
Understanding Airflow Trigger Rules: A Comprehensive Visual Guide Tamara Fingerlin Aug. 06, 2024 1288 -
Introducing Apache Airflow 2.10 Kenten Danas Aug. 16, 2024 1385 -
The Need for Full-Stack Orchestration in the Age of the Data Product Steven Hillion Aug. 19, 2024 547 -
A Musical Interlude: How Orchestras Inspire Modern Data Orchestration Steven Hillion Aug. 22, 2024 917 -
Astro by Astronomer Delivered 438% ROI: Insights from a Forrester TEI Study Astronomer Jan. 31, 2024 423 -
Everything You Need to Know About Apache Airflow® 2.2.0 Julia Wrzosińska Oct. 12, 2021 943 -
The AI Spring: How Demand for Production-Ready GenAI Projects is Continuing to Grow Steven Hillion Aug. 27, 2024 857 -
Data Products: It's not what you call them that matters. It’s what you do with them Jason Ma Aug. 29, 2024 1525 -
Join Us at Airflow Summit 2024: Discover the Future of Apache Airflow with Astronomer Astronomer Sep. 06, 2024 348 -
Customizing LLMs Through Astro Moulay Zaidane Draidia Sep. 06, 2024 896 -
Astro Observe is now in Private Preview: Unifying Data Observability and Orchestration Whit Moses Sep. 10, 2024 565 -
ETL for Beginners: Data Ingestion at Scale with S3 and Snowflake Tamara Fingerlin Sep. 23, 2024 2998 -
Navigating the 2024 Gartner® Market Guide for DataOps Tools: What You Need to Know Astronomer Sep. 24, 2024 1694 -
Astro and Terraform: Empowering Infrastructure as Code for Modern Data Orchestration Isaac Chung Sep. 26, 2024 563 -
Best Practices and Solutions for Multi-Tenant Airflow Viraj Parekh Sep. 27, 2024 1458 -
Introducing Cosmos 1.6: The Best Way to Run dbt-core in Airflow Pankaj Singh Sep. 27, 2024 889 -
Announcing Astronomers Vulnerability Disclosure Program Astronomer Oct. 04, 2024 297 -
Introducing Cohort 4 of the Astronomer Champions Program for Apache Airflow! Briana Okyere Oct. 24, 2024 249 -
Airflow in Action: Data Engineering Insights from Uber and It’s 200,000 Data Pipelines Matthew Keep Oct. 29, 2024 561 -
Unlocking the Power of Scalable Machine Learning with Anyscale and Astronomer Radhika Gulati Oct. 29, 2024 1134 -
Proactive Airflow Monitoring: How to Prevent Infrastructure Issues Before They Happen Stephanie Niu Nov. 05, 2024 920 -
Airflow in Action: Data Engineering Insights from Burns & McDonnell Matthew Keep Nov. 07, 2024 850 -
Airflow in Action: ETL Insights from Bloomberg — Slashing Runtimes by 50% Matthew Keep Nov. 07, 2024 943 -
Airflow in Action: Supercharging Data Science and ML Workflows At Apple Matthew Keep Nov. 14, 2024 782 -
Customer Story: Autodesk’s Data Engineering Transformation with Astronomer and Apache Airflow Astronomer Nov. 15, 2024 893 -
It Pays to be Picky: Leveraging Key Insights with Airflow and Astro Michael Robinson Nov. 15, 2024 2134 -
Airflow in Action: Scaling Insights from Bosch and 1.2 Million Pipeline Runs Per Day Matthew Keep Nov. 19, 2024 489 -
Airflow in Action: Data Engineering Insights from Processing PBs of Data Every Day at Stripe Matthew Keep Nov. 21, 2024 676 -
Airflow in Action: Infrastructure Management Insights from 1 Million Monthly Deploys at LinkedIn Matthew Keep Nov. 26, 2024 533 -
Airflow in Action: ML and LLM Ops Insights from ASAPP — Reducing Workflow Runtimes by 85% Matthew Keep Nov. 27, 2024 845 -

By Matt Makai. 2021-2024.