Company
Date Published
Author
-
Word count
7120
Language
English
Hacker News points
None

Summary

In this blog post, the authors describe how to leverage Argo CD, AWS EKS, and Karpenter to deploy self-hosted GitHub Actions runners and Dagger Engines in a way that streamlines Dagger upgrades in production. The approach is specifically built for horizontal scaling, which means Dagger Engines are ephemeral processes that show up and go away based on queued pipelines. This allows for reduced costs as usage decreases. The authors also discuss the trade-offs of horizontal vs vertical scaling, where vertical scaling involves "monolithic" long-lived Engines that run most of the pipelines. The setup described in the post is currently being used by the Dagger team for all their pipelines, including those from contributions to Dagger. It achieves significant cost savings and improves efficiency in CI job scheduling and Dagger Engine stopping. The authors encourage others to try out this new approach and provide feedback on any issues encountered. They are also working on the third version of Dagger in production and experimenting with running GitHub Actions Runner as a service within Dagger.