Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support HTTPScaledObject scoped timeout #813

Open
Tracked by #911
JorTurFer opened this issue Oct 9, 2023 · 12 comments
Open
Tracked by #911

Support HTTPScaledObject scoped timeout #813

JorTurFer opened this issue Oct 9, 2023 · 12 comments
Labels
help wanted Extra attention is needed stale-bot-ignore All issues that should not be automatically closed by our stale bot

Comments

@JorTurFer
Copy link
Member

Proposal

Currently, there is a timeout configuration for setting the request timeout.
Allowing HTTPScaledObjects to override the timeout for specific route can help to users who have some slow routes but they prefer to not increase the global timeout for all the routes just due to some of them

Use-Case

No response

Is this a feature you are interested in implementing yourself?

Maybe

Anything else?

No response

Copy link

stale bot commented Dec 8, 2023

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale All issues that are marked as stale due to inactivity label Dec 8, 2023
Copy link

stale bot commented Dec 15, 2023

This issue has been automatically closed due to inactivity.

@stale stale bot closed this as completed Dec 15, 2023
@oto-macenauer-absa
Copy link

Hi, is this expected in future releases?

We're using shared k8s cluster so it would be easier to use scoped timeout than asking platform team to come with values that fit all teams sharing the cluster. Also what kinds of timeouts is this expected to support?

@JorTurFer
Copy link
Member Author

Hi, is this expected in future releases?

No, this has been closed automatically due to inactivity. Are you willing to contribute with it?

We're using shared k8s cluster so it would be easier to use scoped timeout than asking platform team to come with values that fit all teams sharing the cluster. Also what kinds of timeouts is this expected to support?

The original proposal was to override the "cold start timeout"

@JorTurFer JorTurFer reopened this Feb 15, 2024
@stale stale bot removed the stale All issues that are marked as stale due to inactivity label Feb 15, 2024
@oto-macenauer-absa
Copy link

No, this has been closed automatically due to inactivity. Are you willing to contribute with it?

Sorry, I don't have any experience with go and this doesn't seem like easy task to start.

The original proposal was to override the "cold start timeout"

Ok, we got API that can take long to answer so I was looking for something to override the responseHeaderTimeout

@JorTurFer
Copy link
Member Author

I've checked and I guess that we could configure all the timeouts at same place from the HTTPScaledObject, so maybe could support all of them.
https://github.com/kedacore/http-add-on/blob/main/interceptor/proxy_handlers.go#L46-L86

@tomkerkhove ?

@tpfau
Copy link

tpfau commented Mar 12, 2024

Having something like this would be great. We were trying to deploy LLMs this way on a cluster, having at most a few requests pending before adding new resources.
But the responses (at least non streamed ones) can take quite some time before they are ready and those are then killed by the 500ms response header timeout and updating this for individual endpoints/pods would be great.

@JorTurFer JorTurFer mentioned this issue Apr 4, 2024
19 tasks
@JorTurFer JorTurFer added the help wanted Extra attention is needed label Apr 4, 2024
Copy link

stale bot commented Jun 6, 2024

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale All issues that are marked as stale due to inactivity label Jun 6, 2024
@zroubalik zroubalik removed the stale All issues that are marked as stale due to inactivity label Jun 6, 2024
@tpfau
Copy link

tpfau commented Jun 25, 2024

I would be willing to have a closer look at this but I have literally no idea where to start (and no experience whatsoever with coding for keda). Any suggestion or example of a variable that has similar behaviors where I could have a look to start it ?

@harsh-viradia
Copy link

I found a way to change the default timeout of keda http addon, while installing http addon via the helm we can change the default value from values.yaml file.

interceptor:
responseHeaderTimeout: 300s
tcpConnectTimeout: 300s
keepAlive: 300s
resources:
limits:
memory: 1Gi
requests:
memory: 512Mi

Copy link

stale bot commented Sep 28, 2024

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale All issues that are marked as stale due to inactivity label Sep 28, 2024
Copy link

stale bot commented Oct 5, 2024

This issue has been automatically closed due to inactivity.

@stale stale bot closed this as completed Oct 5, 2024
@zroubalik zroubalik added stale-bot-ignore All issues that should not be automatically closed by our stale bot and removed stale All issues that are marked as stale due to inactivity labels Oct 18, 2024
@zroubalik zroubalik reopened this Oct 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed stale-bot-ignore All issues that should not be automatically closed by our stale bot
Projects
Status: Done
Development

No branches or pull requests

5 participants