Software Feature Experimentation Tips

Experimentation is crucial in validating new ideas, methods or activities for an application, feature, or infrastructure. It differs from testing as experiments lead to new knowledge while testing only verifies behavior. Common types of experimentation include A/B testing, multi-armed bandits, and chaos engineering. Experiments help validate changes proposed to a product, learn which features impact users the most, iterate and improve on features, and discover what doesn't work. The frequency of experiments depends on the company, but they should be run for major features, site redesigns, significant functionality changes, or performance tweaks. Involvement in experiments varies based on their type, and it is recommended to run them in production with appropriate controls and kill-switches.