Company
Date Published
Nov. 14, 2019
Author
Dawn Parzych
Word count
986
Language
English
Hacker News points
None

Summary

Experimentation is crucial in validating new ideas, methods or activities for an application, feature, or infrastructure. It differs from testing as experiments lead to new knowledge while testing only verifies behavior. Common types of experimentation include A/B testing, multi-armed bandits, and chaos engineering. Experiments help validate changes proposed to a product, learn which features impact users the most, iterate and improve on features, and discover what doesn't work. The frequency of experiments depends on the company, but they should be run for major features, site redesigns, significant functionality changes, or performance tweaks. Involvement in experiments varies based on their type, and it is recommended to run them in production with appropriate controls and kill-switches.