Introducing Cosmos 1.6: The Best Way to Run dbt-core in Airflow
Cosmos 1.6.0 has been released, introducing new features and enhancements for the community. The update allows users to integrate dbt-core jobs into Airflow by automatically creating Airflow tasks from dbt models. Key additions include a new way of rendering source nodes as Airflow tasks through the RenderConfig API, persisting the virtual environment directory for LoadMode.VIRTUALENV, and better support for using remote manifest files in GCS/S3/Azure Blob Storage. Additionally, Cosmos 1.6 supports caching dbt package-lock.yml file to ensure predictable installation of project dependencies and enhance performance.
Company
Astronomer
Date published
Sept. 27, 2024
Author(s)
Pankaj Singh
Word count
889
Language
English
Hacker News points
None found.