-
Notifications
You must be signed in to change notification settings - Fork 554
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Optimize get_artifacts_for_build() with regex filtering #4297
base: master
Are you sure you want to change the base?
Conversation
/gcbrun |
target=target, | ||
attemptId=attempt_id, | ||
nameRegexp=regexp, | ||
maxResults=100) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is maxResults passed here but not above?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The maxResults=100 parameter is passed in the else block to ensure that all artifacts matching the provided regular expression are retrieved in a single request. This is because the default maximum result limit of 9 artifacts might be insufficient when using a regex that matches a large number of artifacts.
In earlier approach(if-case) , there were no regex based filtering and we were fetching artifacts name one by one using buildartifact.list() api resulting in more number of api calls. and no need of passing 'maxResults' argument.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This change looks good.
@aditya-wazir it would be really interesting to see about adding a test for files located in the src/clusterfuzz/_internal/platforms/android/
directory. As far as I can tell, there are not any (at least, not in src/clusterfuzz/_internal/tests/core
).
Could you please investigate the level of effort to add a new test file for android here? Maybe we could just start with a single fetch_artifact_test.py
file, and build on the structure thereafter.
Introduces regex-based filtering directly in the API to improve performance and reduce number of calls to ab api server.
b115c53
to
36657a2
Compare
Hi Mark, |
/gcbrun |
Introduces regex-based filtering directly in the API to improve performance and reduce number of calls to ab api server.