Anatomy of a Throttler, part 3

In this conclusion of a three-part blog series on throttlers, we discuss client identities, cooperation, prioritization, and constraints issues. Clients should identify themselves to the throttler for analysis or investigation purposes, as well as operational purposes such as prioritizing one job over others or tuning down specific clients. We propose a safer way of playing within the rules by considering prioritization or de-prioritization through a dice roll mechanism. Throttling on different metrics can be beneficial but should be done with practical considerations in mind. Exemption scenarios may make sense in certain situations, and categorization and breakdown of client identities can help in analyzing throttler access patterns by categories of requests. Lastly, we emphasize the importance of dynamic control of the throttler and the ability to prioritize or push back some requests or jobs in production systems.