Orchestration gets to a weird state

[anhhnguyen206]

Hi there,

I'm not sure if this is a bug or not so submitting as a question first.

I encountered a weird situation when trying to start an orchestration. I can see in the Instances table a new orchestration row has been created but almost all the values are null and there are no rows in History table. Unsure how it got to that state but our system relies heavily on starting up orchestrations for long running work and it's the first time I've seen this.

The instance id is: 9876465ccb0b44278132cf5e42b6e8e3:0

Appreciate any insights!

[cgillum]

When starting an orchestration, we expect a row in the Instances table that includes the instance ID, the name, the created timestamp, the runtime status (which should start out as "Pending"), and maybe a few others. Having no rows in the History table is expected - those don't get written until the orchestration competes its first checkpoint. There should also be a message in one of the control queues, which is used to start the orchestration's execution (it may disappear quickly though if it gets dequeued right away).

If you're not even seeing the basic values in the Instances table, then something very fundamental may be broken, though I don't have any good ideas about what it may be. I've never heard of such a thing. Can you share a screenshot?

[anhhnguyen206]

Hi @cgillum

Thanks for the quick response.

Here is the screenshot on the current state that I shared.

[cgillum]

That's very odd. By chance do you have any global serialization settings being configured in your app? Recently we saw similarly strange behavior like this because a user used the JsonConvert.DefaultSettings API and accidentally broke our internal serialization logic.

[anhhnguyen206]

We handle our serialization in place so we don't actually use any global settings. It only happened with this particular instance.

We also use 2.4.1 which might be a little bit outdated.

[davidmrdavid]

@anhhnguyen206: it's probably best to update to the latest DF Extension just in case this issue isn't reproducible there. Could you please upgrade and let us know?