Start Orchestrator Function from another Orchestrator Function (Fire and Forget) python

[kanupriya15025]

Below is a sample Eternal Function that I have written.
Note the line "yield context.call_sub_orchestrator()". This line waits for the sub orchestrator to return and then pass control to the next line. Is it possible that I just call the sub orchestrator and forget it. I don't want to track the progress of it from my Eternal Function.

      def orchestrator_function(context: df.DurableOrchestrationContext):

          inputBody = context.get_input()
          workflowId = inputBody.get('name')
          schedule = inputBody.get('parameters').get('schedule')
          local_date = context.current_utc_datetime
          iter = croniter(schedule, local_date)
       
          #RunId will have to generated and passed here
          date_time = local_date.strftime("%m:%d:%Y:%H:%M:%S")
          instanceId = workflowId + "_" + date_time
          
          if not context.is_replaying:
              logging.info('Calling Orchestrator for the workflow: {} with runId {}'.format(workflowId, instanceId))
      
          yield context.call_sub_orchestrator("WorkflowOrchestrator", inputBody, instanceId)
      
          next_wait_time = iter.get_next(datetime)
          if not context.is_replaying:
              logging.info('If updated later : Waiting till {}'.format(next_wait_time))
      
          yield context.create_timer(iter.get_next(datetime))

          context.continue_as_new(inputBody)
      
      main = df.Orchestrator.create(orchestrator_function)

[davidmrdavid]

Hi @kanupriya15025, thanks for reaching out.

Our yield'able APIs unfortunately cannot be invoked as fire-and-forget, and that means that call_sub_orchestrator is a blocking operation. However, you should be able to start an orchestrator in a fire-and-forget way, but it needs a bit more work. The easiest path would be to replace your call_sub_orchestrator call with a call_activity. In that Activity, you can request a DurableOrchestrationClient object that you may use to start an orchestrator through client.start_new. Does that make sense?

As with any Activity, do recall that these may be called more than once, so it's important that you start your orchestrator in an idempotent fashion. You can do this in your Activity by first verifying that the orchestrator instanceID doesn't currently exist. This can be done via client.get_status. This documentation more-or-less illustrates the pattern: https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-singletons?tabs=python#singleton-example

Please let me know if you need more guidance. Thanks!

[kanupriya15025]

So, just to iterate if I understand it right.
You are saying that the eternal function should call an activity function in a fire and forget manner. And then this Activity function should be responsible for tracking the sub-orchestrator. Is it right to assume that the activity function's run time will be equal to sub-orchestrator execution time?

[kanupriya15025]

And yeah, thanks for such a quick response :)

[davidmrdavid]

@kanupriya15025: Hmm, some of the details aren't quite right. Let me try to explain.

To be clear: you cannot call an Activity Function in a fire and forget manner. You will have to wait for the Activity to return a value.

However, the Activity does not need to wait for the orchestrator to run to completion. This is the key.
The Activity can simply run so long as its necessary to start the orchestration, and then return while the orchestration is still running.

This is "effectively" fire and forget because it fires the orchestration, and ignores its result. But it is not fire and forget in the strict sense, because you do need to wait for the Activity to return, which it will as soon as it starts the orchestration.

Does that make a bit more sense?

[kanupriya15025]

Hey David,
This works perfected as intended. Thanks for helping me out here!

[kanupriya15025]

This is the code that I improved :
# yieldcontext.call_sub_orchestrator("WorkflowOrchestrator", inputBody, instanceId)
output = yield context.call_activity("CallSubOrchestratorAsync", inputBody)

And the activity function looks like this. :

existing_instance = await client.get_status(instance_id)

    if existing_instance.runtime_status in [df.OrchestrationRuntimeStatus.Completed, df.OrchestrationRuntimeStatus.Failed, df.OrchestrationRuntimeStatus.Terminated, None]:
        logging.info('Trying to create an instance with id: {}'.format(instance_id))
        instance_id = await client.start_new("WorkflowOrchestrator", instance_id, inputBody)
        return {
            'status': 201,
            'body': f"Instance started with ID '${instance_id}' already exists"
        }
    else:
        return {
            'status': 409,
            'body': f"An instance with ID '${instance_id}' already exists"
        }