[SUGGESTION] Activity Per Instance Option

[arman-g]

I think, it would be a good idea to have an option in TaskOptons of Activity that will act as "maxConcurrentActiveityFunctions": 1 global configuration option but for that particular Activity. This new option/flag would be very useful when some activities should run under their own dedicated instance.

Here is our use case where this option would be very effective:
We use Azure Durable Function App to manipulate (merge, add pagination etc.) and also run OCR process to produce searchable PDF files (Court Transcripts). Since the OCR process is very memory intensive, we configured the activities to run on a single instance by having "maxConcurrentActiveityFunctions": 1 in the host.json. This works well but it is very costly since the other Activities (e.g. assemble, update DB, clean up etc.) also run in their own instances instead of sharing one.

If considered, the option should override the global configuration "maxConcurrentActiveityFunctions" for the Activity.

The same can be applied to Orchestration by adding the option under the "StartOrchestrationOptions".

[cgillum]

We're currently working through some plans to enable splitting an orchestration across multiple function apps, which would allow you to (among other things) have an app that runs on dedicated compute with its own concurrency configuration, separate from the "main" orchestration app. I think that would solve the issue you're trying to solve through this feature suggestion. I can't go into too many more details now, but more will be announced later.

[vigouredelaruse]

re: have an app that runs on dedicated compute with its own concurrency configuration, separate from the "main" orchestration app

i'm implementing something that seems to be one of your target use cases as in

• a multi-tenant cloud native (k8s hosted) gis app that must minimize the etl costs (bandwidth, storage) of large transient datasets (as per the diagram)
• implemented with data flow as per | asp.net core <--> queue <--> orchestrations <--datasets
• network participants may be 'remote' analysts, students and educators who can use their own compute, network and storage without causing bill back to their institutions/businesses

the intention is to move transient high bandwidth data flows and storage to 'user hosted' infrastrastructure, with only analysis results persisted to the cloud

the main challenge in this scenario is how to minimize the required infrastructure surface that must be exposed to remote low trust compute - looking forward to the details you teased as well as timely ability to move my in-proc orchestrations to .net 8

thanks