I Don't Always Test My Code, But When I Do It's In Production

Isaac Mosquera, CTO at Armory.io, discussed canary deployments and their role in increasing development productivity, reducing risk, and keeping developers happy. He explained the concept of Canaries as a method to test new software versions in production by deploying one node into production, assessing its performance, and then scaling out if it performs well or rolling back if not. Automated canary analysis helps eliminate human bias from the decision-making process during deployment. Mosquera emphasized that while automated canaries are helpful, they should be used in conjunction with other testing methods such as unit tests and integration tests to ensure a safe and efficient software delivery system. He also shared lessons learned from implementing Canaries, including using simple statistics instead of machine learning for better transparency, creating separate baseline clusters for accurate comparisons, and choosing the right metrics based on each application's profile. The talk concluded with an overview of Armory's Spinnaker platform, which automates canary analysis and provides a visual representation of pipeline stages, enabling developers to monitor and control their deployments effectively.