Support multiple categories of Orchestrator to tweak polling

[SimonLuckenuik]

Note: This is a suggestion related to #618 and #607

I think it would be interesting to be able to have "orchestrator categories" inside same Function App to fine tune even more the cost based on usage scenario.

For example, we have some Orchestrators which are basically timer based with long timers (days to months), sitting idle most of the time, so every minute polling (maybe more, like 5 minutes) would probably do the trick and lower the idle cost a lot. We also have some other Orchestrators in the same Function App which are more realtime and would maybe require polling at every 500ms.

Without going into flexible custom categories, something like Low priority, Medium priority and High priority orchestrators with specific configuration for each of those would probably do the trick.

For that suggestion to make sense, it would require to have the polling interval adjust dynamically based on active orchestrators (if only Low priority orchestrators active, poll every 5 minutes, if at least one High priority is active, poll every 200ms, once completed fallback to every 5 minutes).

[cgillum]

I think in order to justify designing this kind of a system (which would be difficult since the queue infrastructure is shared for all orchestration definitions), we would need to answer the following questions:

• How much value is added by having this feature vs. having three different function apps, one for each category (low, medium, and high)?
• What kind of cost savings do we think this will provide relative to how things work today, with shared, static polling configuration?

I think having multiple function apps with different configurations might be a very reasonable approach. We're doing work to allow multiple function apps to share the same history and instances tables, so that might help reduce the friction of such an option.

[SimonLuckenuik]

I don't know the details of Durable Functions and related frameworks, but from an implementation perspective it doesn't necessarly mean to rebuild everything from scratch or involve major changes. It could mean to have 3 sets of queue infrastructure and 3 sets of orchestration (one set per priority), but rather have the same concept executing multiple times using different resources. Another option would be to be more "reactive", Durable Function is already coupled with Event Grid for notifications of the state changes, so using Event Grid to send events that would wake-up the orchestrators to start polling could be an option as well, that would be an opt-in to allow to have less frequent polling but react quickly to changes.

For the first point, it all depends how work is organized, We try to limit the number of Function Apps, simply from a devops and monitoring overhead perspective (more builds, more deployments, more infrastructure deployment). We keep one business domain per service, so it would me duplicating the services (and related configuration) just to optimize polling costs of Durable function based on the business scenarios, not attractive from a development point of view.

For the second point, I guess it depends on the scenarios and target pricing for our solution. For us, out of 5-6 different workflows, probably 2-3 are using long timers (days to months), all in different services. The other workflows have short lives and would require quicker polling, only while at least one workflow is alive (scenarios like map-reduce to generate reports and such).

While your suggestion to duplicate app would work, from a developer experience, it is not great.