Add rate limiter primitives #235
Conversation
| ): | ||
| /** Limits the rate of execution of the given operation | ||
| */ | ||
| def apply[T](operation: => T): Option[T] = |
There was a problem hiding this comment.
do I understand correctly that the result is None when the limit is exceeded, and the policy is Drop? and Some(_) when the limit is not exceeded, or it is exceeded, but the policy is to Block?
If so, I think we'd have to split this into two operations: RateLimiter.runBlocking(t: T): T and RateLimiter.runOrDrop(t: T): Option[T]. There definitely are scenarios for both policies, but the most basic use-case is to run an operation and block, if the limit is exceeded. If you know that the policy is Block, you'd have to always .get the returned Option, which is bad.
There was a problem hiding this comment.
Thanks for the review! That was the intended behaviour. The problem with splitting is that it would render new BlockingPolicy implementations difficult to make. For example, users might want a custom policy where the operation is just slowed down or blocking the first and dropping the rest. A possibility would be continuing with just apply by making RateLimiter generic:
RateLimiter[F[_]]:
def apply[T](operation: => T): F[T]
I also think that a good idea would be to allow running with a particular configuration, so the final API would be this. Actually we could use dependent types:
RateLimiter(config: RateLimiterConfig):
def apply[T](operation: => T): config.F[T]
def apply[T](operation: => T, opCfg: Cfg): config.F[T]
This would allow a custom BlockingPolicy to implement blocking or dropping (or something different like throttling) behaviour per operation:
rateLimiter(operation, CustomCfg.block())
rateLimiter(operation, CustomCfg.drop())
Only disadvantage might be verbosity but I believe the possibility of custom implementations outweighs it. What are your thoughts before proceeding?
There was a problem hiding this comment.
This could work, but I'm afraid would be too complicated. You're right that we might loose some flexibility, but the main goal would be to address to most-common use-cases - which have to be served well. To be honest, cases such as slowing down the first operation / dropping the rest, seem to be quite specialised, and it would be totally acceptable for them to require writing some custom code. That is, you could reuse the algorithm part, but everything around it would need to be written by hand.
So I'd opt for a simple interface (no dependent / higher-order types) solving the common case, while providing building blocks for implementing more advanced use-cases
| if config.blockingPolicy.isUnblocked then | ||
| if config.algorithm.isUnblocked then | ||
| if config.isReady then | ||
| config.acceptOperation |
There was a problem hiding this comment.
I didn't dive into the implementation yet, but isn't this race'y? That is, if two threads concurrently proceed through the three if-s, they could both concurrently call .acceptOperation, even if this would exceed the limit? It feels like accepting should be an atomic operation, which might fail (due to other threads exceeding the limit)
| scope.shutdown() | ||
| scope.join().discard | ||
| // join might have been interrupted | ||
| try f(using capability) |
There was a problem hiding this comment.
was this reformatted by accident? doesn't look fine, maybe we need braces
|
Thanks for the PR! I left some initial comments. Once these are resolved I'll do a more thorough review. One thing that's missing and that we'd definitely need is some documentation (in |
Implements a customizable rate limiter. The behaviour depend on a
RateLimiterConfigbuilt from aBlockingPolicyand aRateLimiterAlgorithm.BlockingPolicyshould deal exclusively with the response to rejected operations whileRateLimiterAlgorithmmust control only whether an operation can be accepted or not.Currently, there are two blocking policies:
BlockandDrop.Block: If the algorithm gets blocked, new operations will be queued so that when the algorithm gets unblocked, these operations will be processed first.Drop: Operations passed to the rate limiter when the algorithm is blocked will be discardedThere are also 4 algorithm implementations: fixed rate, sliding window, leaky bucket and token bucket.
Both
BlockingPolicyandRateLimiterAlgorithmpresent an interface (which I hope is not confusing) that makes very easy to implement new behaviour. If the guidelines for implementation are followed, things like throttling operations or blocking a particular number and discarding thereafter should be very easy to build.Tests include behaviour for all the 8 different combinations and test the behaviour also in a concurrent context.
/claim #120
fixes #120