Optimize get_artifacts_for_build() with regex filtering #4297
Conversation
|
/gcbrun |
| target=target, | ||
| attemptId=attempt_id, | ||
| nameRegexp=regexp, | ||
| maxResults=100) |
There was a problem hiding this comment.
Why is maxResults passed here but not above?
There was a problem hiding this comment.
The maxResults=100 parameter is passed in the else block to ensure that all artifacts matching the provided regular expression are retrieved in a single request. This is because the default maximum result limit of 9 artifacts might be insufficient when using a regex that matches a large number of artifacts.
In earlier approach(if-case) , there were no regex based filtering and we were fetching artifacts name one by one using buildartifact.list() api resulting in more number of api calls. and no need of passing 'maxResults' argument.
There was a problem hiding this comment.
This change looks good.
@aditya-wazir it would be really interesting to see about adding a test for files located in the src/clusterfuzz/_internal/platforms/android/ directory. As far as I can tell, there are not any (at least, not in src/clusterfuzz/_internal/tests/core).
Could you please investigate the level of effort to add a new test file for android here? Maybe we could just start with a single fetch_artifact_test.py file, and build on the structure thereafter.
Introduces regex-based filtering directly in the API to improve performance and reduce number of calls to ab api server.
aditya-wazir
force-pushed
the
optimize_ab_api
branch
from
b115c53
to
36657a2
Compare
Hi Mark, |
|
/gcbrun |
Introduces regex-based filtering directly in the API to improve performance and reduce number of calls to ab api server.