New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SYNPY-1533] Lower the HTTPX connection limit to 5 #1142

Draft

BryanFauble wants to merge 1 commit into develop from SYNPY-1533-lower-httpx-connection-limit

Contributor

BryanFauble commented Oct 7, 2024 •

edited

Loading

https://sagebionetworks.jira.com/browse/SYNPY-1533

Problem:

Found in this service desk ticket: https://sagebionetworks.jira.com/browse/SYNSD-1233 a collaborator was having issues uploading files for connection/read failures during the upload process.
The default HTTPX library max_connections is 100, for non-storage operations I originally lowered this to 25 as I found this to not affect any performed. This has been further lowered to 5.

Solution:

Lower max_connections to 5 for both storage operations (Like connecting to S3) and Synapse operations (REST API servers)

Testing:

We will need to test on a flaky connection as well as re-run a portion of the download/upload benchmarking to assess performance of the change.


Lower the HTTPX connection limit to 5

8438f85

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment