Extending Buildkite with plugins: HashiCorp Vault

In this blog post by Michael Belton on August 9, 2023, the author discusses extensibility in CI/CD products and how Buildkite supports it through plugins. Plugins are self-contained pieces of functionality that can be added to pipelines to customize Buildkite according to specific workflows. They modify job lifecycle hooks such as setting up environments, checking out code, running commands, handling artifacts, and cleaning up environments. Buildkite plugins can be open source or private, with the latter being accessible only by an organization's agents. The management of plugins is done directly in pipeline definitions rather than a web-based plugin management system. Plugins are used in pipeline command steps to access libraries of commands or perform actions. The Vault secrets plugin is introduced as the recommended way to integrate with HashiCorp Vault, which is an identity-based secrets and encryption management system. The plugin allows agents to authenticate to Vault and acquire pipeline secrets while running a job, enabling more granular policies for access by agents and pipelines. The author also provides examples of how to use the Vault secrets plugin in a pipeline definition and mentions that many common problems or workflows already have dedicated plugins from Buildkite or the Buildkite community. The post concludes with information on where to find existing plugins and how to write custom ones if needed.