How we built a Ruby library that saves 50% in testing time

Long, unstable CI pipelines can be a major issue for many software development teams. One approach to speed up test suites is by running tests in parallel, but this doesn't solve the problem of flaky tests. Another solution is selective testing, which involves dynamically generating a map between each test and the source code files executed during the test. This can result in faster pipelines, fewer cloud resources used, and a smaller chance of unrelated test flakiness. Datadog built an Intelligent Test Runner product around this idea, using a library that collects impacted source code files dynamically during test runs. The tool has a median performance overhead of 25 percent (up to 70 percent for a worst-case scenario), which results in about 5–10 times better performance than existing solutions tried.