Query regarding QoSOptions implemention in version 23.2+

[ankits23560]

Hello everyone,

We are currently experiencing difficulties while trying to implement QoS with Polly V8 in version 23.3.3.
The Issue we are facing is explained below:

Under Polly V7 version 23.2.2:

JSON Configuration:

"QoSOptions": {
    "ExceptionsAllowedBeforeBreaking": 3, // Number of retries before circuit breaker opens
    "DurationOfBreak": 15000, // Duration in milliseconds the circuit breaker will be open, sending back a 503
    "TimeoutValue": 5000 // Duration in milliseconds waiting for a response before a timeout occurs
}

Program. cs Configuration:

services.AddJwtAuthentication();
services.AddOcelot().AddPollyV7();

Behavior Of QoSOptions with Polly V7:
On Implementing QoSOptions with Polly V7 we observed that the circuit breaker was breaking the circuit when the Exceptions occurred more than 3 times and also breaking the circuit when the timeout exceptions exceeded 3 times. However, on implementing QoS with Polly v8 the circuit is breaking the circuit with the exceptions but not breaking the circuit when the timeout exceptions.

We would appreciate your help in generating the scenario mentioned above for implementing the QoS circuit breaker with Polly V8 in version 23.3.3 since it is not clearly explained in the Polly documentation.

Is it possible to implement the circuit breaker strategy and the timeout strategy together?

Is there any workaround regarding this issue?

Also, How can we implement the QoS behavior of Polly v7 in Polly v8?
Also Requesting can you please update the documentation for Ocelot regarding this issue.
Thanks
Ankit Sukhwani

[ankits23560]

hi @RaynaldM can you please guide me on the same?

[raman-m]

In @RaynaldM we trust! 😄

[ankits23560]

Hi @raman-m, could you please provide any inputs that I can try?

[raman-m]

@ankits23560 Read my answer below plz! 👇

[raman-m]

Hello Ankit!

Behavior Of QoSOptions with Polly V7:

I'm not sure why you're researching the previous version 23.2 vs Polly v7. It would be better to migrate to the new version 23.3 vs Polly v8. Our team's intentions have been explained in the Ocelot extra packages section of the Release Notes.
Support for version 23.2 will be discontinued. Our release strategy focuses on developing new versions without maintaining older ones.

---

On Implementing QoSOptions with Polly V7 we observed that the circuit breaker was breaking the circuit when the Exceptions occurred more than 3 times and also breaking the circuit when the timeout exceptions exceeded 3 times. However, on implementing QoS with Polly v8 the circuit is breaking the circuit with the exceptions but not breaking the circuit when the timeout exceptions.

It's unusual! We already manage timeout exceptions in the configuration of the Circuit Breaker strategy:

Ocelot/src/Ocelot.Provider.Polly/PollyQoSResiliencePipelineProvider.cs

Lines 92 to 93 in 8c0180a

	.Handle<TimeoutRejectedException>()
	.Handle<TimeoutException>(),

Can you specify which exceptions you're referring to? What are the exact types?

Additionally, we log Information messages at this location:

Ocelot/src/Ocelot.Provider.Polly/PollyQoSResiliencePipelineProvider.cs

Line 130 in 8c0180a

_logger.LogInformation($"Timeout for Route: {GetRouteName(route)}");

Could you modify the logging settings to utilize the Information level to view the provider's messages?
Please examine your logs for specific messages and provide us with the complete request log.

---

We would appreciate your help in generating the scenario mentioned above for implementing the QoS circuit breaker with Polly V8 in version 23.3.3 since it is not clearly explained in the Polly documentation.

The details are outlined in our Quality of Service documentation. However, it seems you might have some customizations for the QoS feature, correct? Without seeing your solution, it's challenging to offer specific advice. If your scenario involves both strategies, please review the Configuration section again.

I'm concerned that your configuration's DurationOfBreak is three times greater than the TimeoutValue, with 15000 being more than three times 5000. This configuration implies that most requests will time out when the Circuit is open. Moreover, if the Circuit is closed, the Circuit Breaker strategy will absorb timeout exceptions due to the default settings.

Should you require behavior that deviates from the default, you'll need to create your own provider utilizing the Extensibility feature.

---

Is it possible to implement the circuit breaker strategy and the timeout strategy together?

Indeed, it is possible to implement both the circuit breaker and timeout strategies concurrently. For guidance, consult the Configuration section in the documentation. Remember, you must define all three properties to activate both strategies.

---

Is there any workaround regarding this issue?

To address this issue, I suggest the following:

• Set DurationOfBreak to be equal to or less than TimeoutValue. It is generally advised that TimeoutValue be significantly higher than DurationOfBreak.
• Activate the Information logging level and conduct further investigation. This lower level of logging should provide sufficient details to comprehend the behavior of your QoS configuration.
• Don't use AddPollyV7 and version 23.2.x! Use AddPolly and version 23.3+!!!

---

Also, How can we implement the QoS behavior of Polly v7 in Polly v8?

Seems you have to implement your own QoS provider using the Extensibility feature.
Please learn official services helpers C# code:

• Polly QoS OcelotBuilderExtensions

Keep in mind that we no longer support Polly v7❗ The Polly library released version 8 in the previous year, 2023. It makes no sense to develop using Polly v7, which is why we have removed the v7 logic and provider.

---

Also Requesting can you please update the documentation for Ocelot regarding this issue.

Our official documentation already contains sufficient information and links to Polly for implementing QoS in Ocelot solutions.

Hope it helps!

[ggnaegi]

@raman-m We should get rid of the Timeout Delegating Handler... I told you that a few months ago :-)

[raman-m]

@ggnaegi
After removing that, will the QoS stabilize? It appears this is your design and logic. I apologize, but I don't recall our previous discussions on this matter. Could you provide any links? I only remember that we have discussed this.

Regarding the next question, the TimeoutDelegatingHandler class is pivotal to our timeout feature. If we eliminate this handler, what will be our method for configuring timeouts? Will it be based on HttpRequestMessage? Also, we no longer utilize HttpClient in the kernel.
As for now, I don't see an alternative for timeout feature.

Should we convert this discussion into an issue, or would you prefer to continue in this thread? Please let me know when you will be available to work on this.

[raman-m]

I apologize, @ggnaegi. Are you suggesting that the handler ought to be removed so that QoS routes can be managed by the TimeOut strategy, while the handler should be retained for regular non-QoS routes to be managed by MessageInvokerPool?

[ankits23560]

hi @ggnaegi it seems that this is an issue. We are facing major problems in implementing circuit breaker with timeout-strategy.
Is there any workaround going on regarding this thread?

[raman-m]

@ankits23560 Dear Ankit,
It appears that our QoS logic in version 23.3 does not meet your needs.
I suggest disabling the QoS feature entirely and creating your own delegating handler where you can implement the previous Polly v7 logic or migrate to v8 seamlessly. Our team plans to phase out the TimeoutDelegatingHandler in the near future, possibly in the forthcoming releases.

P.S.

Please review note 1 and inform us if the exception has been resolved in your logs. Thank you for your understanding.

P.P.S.

Please be aware that we have an open PR #2073, which involves a redesign of the Timeout feature. This PR will have implications for Quality of Service (QoS).

[ankits23560]

Hi @raman-m,
can we move this discussion to the issue? We still have no confirmation about the issue?

[raman-m]

Please display the issue in the open pull request. It does not matter whether you attach the PR to the discussion thread or to an issue thread.

[raman-m]

👉 #2112 (reply in thread)

[raman-m]

@ggnaegi wrote on Jul 19, 2024

We should get rid of the Timeout Delegating Handler... I told you that a few months ago :-)

Alright, Gui, you told me here... I now have a clearer understanding of the timeout logic 😇 and it appears we can replace our custom TimeoutDelegatingHandler with the SocketsHttpHandler class. Does that sound good?
Regarding the delivery of #2073, should it be before or after the removal of TimeoutDelegatingHandler?
It seems we should deliver #2073 first, followed by the removal of TimeoutDelegatingHandler, as it requires significant refactoring of the core messaging logic, correct?

I propose the following task →

TODO: Implement Timeout Logic Using the SocketsHttpHandler Class

We need to utilize the following properties:

• ConnectTimeout
• Expect100ContinueTimeout
• KeepAlivePingTimeout
• PooledConnectionIdleTimeout
• ResponseDrainTimeout

[raman-m]

Added the task to TODOs aka Follow-up tasks