Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow configuring timeout for external sources #1812

Open
wants to merge 43 commits into
base: main
Choose a base branch
from

Conversation

pouyan021
Copy link

@pouyan021 pouyan021 commented Apr 18, 2024

This pull request closes #1624. It adds and enforces the ability to set a new property abort-after to external sources. As discussed in the issue, it supports both a global prop and a maven property that overrides the global if it is set.

spiffcs
spiffcs previously approved these changes Apr 22, 2024
README.md Show resolved Hide resolved
@spiffcs
Copy link
Contributor

spiffcs commented Apr 22, 2024

@pouyan021 Approved and running checks now - Thank you so much for the contribution!

@spiffcs
Copy link
Contributor

spiffcs commented Apr 22, 2024

@pouyan021 looks like there is a small change needed where go mod tidy is run

@pouyan021 pouyan021 force-pushed the feat/allow-configuring-timeout-for-external-sources branch 2 times, most recently from 5cc6f4b to e5ed88d Compare April 22, 2024 17:46
@pouyan021
Copy link
Author

pouyan021 commented Apr 22, 2024

@spiffcs Appreciate your support, thanks a lot! The problem with go.mod is addressed but I forgot to sign-off my commit! I fixed that and pushed again. Could you kindly approve the checks once more?

@pouyan021 pouyan021 requested a review from spiffcs April 25, 2024 06:19
Copy link
Contributor

@spiffcs spiffcs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything else LGTM - Just waiting for @wagoodman and his final say on the config direction he wants to go here

base-url: https://search.maven.org/solrsearch/select
abort-after: 5m #override the global config
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @wagoodman - I know he's pretty sensitive to duplicate fields that override each other so I'd like him to chime in on where he sees this going or what his preference would be

@spiffcs spiffcs self-requested a review April 25, 2024 18:34
@spiffcs spiffcs dismissed their stale review April 25, 2024 18:35

stale review and waiting for IC input

@wagoodman wagoodman changed the title Feat/allow configuring timeout for external sources Allow configuring timeout for external sources May 16, 2024
Copy link
Contributor

@wagoodman wagoodman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall the functionality looks good, but have some comments on testing and configuration. I'll push up some changes shortly to help out.

I do think that RequestTimeout is a better name for this config item -- what do you think @pouyan021 ?

cmd/grype/cli/options/datasources.go Outdated Show resolved Hide resolved
grype/matcher/java/matcher_mocks_test.go Outdated Show resolved Hide resolved
grype/matcher/java/matcher_mocks_test.go Outdated Show resolved Hide resolved
@@ -278,9 +278,11 @@ feature is currently disabled by default. To enable this feature add the followi
```yaml
external-sources:
enable: true
abort-after: 10m
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can clarify what this means by changing the name some. This could be interpreted as either:
a. aborting looking up from external sources in general after the duration elapses
b. aborting a single request to an external source after the duration elapses

From the functionality implemented b is implied.

Regarding naming and the above context, request-timeout feels like a more descriptive name.

cmd/grype/cli/options/datasources.go Outdated Show resolved Hide resolved
@wagoodman wagoodman force-pushed the feat/allow-configuring-timeout-for-external-sources branch from ca5d50c to 0bf64dd Compare May 16, 2024 15:29
@wagoodman
Copy link
Contributor

note: I force pushed to get this branch rebased onto the latest commit on main

@pouyan021
Copy link
Author

pouyan021 commented May 17, 2024

Overall the functionality looks good, but have some comments on testing and configuration. I'll push up some changes shortly to help out.

I do think that RequestTimeout is a better name for this config item -- what do you think @pouyan021 ?

Hey @wagoodman thanks a lot for the thorough review and your additional changes. The name was chosen as abort-after based on the suggestion by one of the contributors here. I agree with you that RequestTimeout is more self-explanatory.

@pouyan021
Copy link
Author

Hey @wagoodman should I go for the requestTimeout as per your suggestion?

@pouyan021
Copy link
Author

Hey @wagoodman @spiffcs I resolved the conflict on this branch recently, any plans for this to move forward?

@spiffcs
Copy link
Contributor

spiffcs commented Jul 2, 2024

@pouyan021 I'm so sorry here - I've approved and have run the final checks and will get this merged - I did not see the notification for this and apologize for letting it sit

I'll get the commits in the wrap this up 😄

dependabot bot and others added 4 commits September 17, 2024 11:15
…#1871)

Bumps [anchore/sbom-action](https://github.com/anchore/sbom-action) from 0.15.11 to 0.16.0.
- [Release notes](https://github.com/anchore/sbom-action/releases)
- [Commits](anchore/sbom-action@7ccf588...e8d2a69)

---
updated-dependencies:
- dependency-name: anchore/sbom-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…#1870)

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2.13.4 to 3.25.6.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@cdcdbb5...9fdb3e4)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: westonsteimel <[email protected]>
Bumps [actions/checkout](https://github.com/actions/checkout) from 4.1.5 to 4.1.6.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@44c2b7a...a5ac7e5)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…4.4 (anchore#1889)

Bumps [github.com/gabriel-vasile/mimetype](https://github.com/gabriel-vasile/mimetype) from 1.4.3 to 1.4.4.
- [Release notes](https://github.com/gabriel-vasile/mimetype/releases)
- [Commits](gabriel-vasile/mimetype@v1.4.3...v1.4.4)

---
updated-dependencies:
- dependency-name: github.com/gabriel-vasile/mimetype
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [github.com/charmbracelet/bubbletea](https://github.com/charmbracelet/bubbletea) from 0.26.2 to 0.26.3.
- [Release notes](https://github.com/charmbracelet/bubbletea/releases)
- [Commits](charmbracelet/bubbletea@v0.26.2...v0.26.3)

---
updated-dependencies:
- dependency-name: github.com/charmbracelet/bubbletea
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: westonsteimel <[email protected]>
…anchore#1887)

Bumps [github.com/hashicorp/go-version](https://github.com/hashicorp/go-version) from 1.6.0 to 1.7.0.
- [Release notes](https://github.com/hashicorp/go-version/releases)
- [Changelog](https://github.com/hashicorp/go-version/blob/main/CHANGELOG.md)
- [Commits](hashicorp/go-version@v1.6.0...v1.7.0)

---
updated-dependencies:
- dependency-name: github.com/hashicorp/go-version
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [github.com/charmbracelet/lipgloss](https://github.com/charmbracelet/lipgloss) from 0.10.0 to 0.11.0.
- [Release notes](https://github.com/charmbracelet/lipgloss/releases)
- [Commits](charmbracelet/lipgloss@v0.10.0...v0.11.0)

---
updated-dependencies:
- dependency-name: github.com/charmbracelet/lipgloss
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* update to latest syft

Signed-off-by: Alex Goodman <[email protected]>

* fix tests related to syft bump

Signed-off-by: Alex Goodman <[email protected]>

---------

Signed-off-by: Alex Goodman <[email protected]>
Bumps [docker/login-action](https://github.com/docker/login-action) from 3.1.0 to 3.2.0.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](docker/login-action@e92390c...0d4c9c5)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: westonsteimel <[email protected]>
Bumps [github.com/charmbracelet/bubbletea](https://github.com/charmbracelet/bubbletea) from 0.26.3 to 0.26.4.
- [Release notes](https://github.com/charmbracelet/bubbletea/releases)
- [Commits](charmbracelet/bubbletea@v0.26.3...v0.26.4)

---
updated-dependencies:
- dependency-name: github.com/charmbracelet/bubbletea
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…#1901)

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.25.6 to 3.25.7.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@9fdb3e4...f079b84)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [github.com/docker/docker](https://github.com/docker/docker) from 26.1.3+incompatible to 26.1.4+incompatible.
- [Release notes](https://github.com/docker/docker/releases)
- [Commits](moby/moby@v26.1.3...v26.1.4)

---
updated-dependencies:
- dependency-name: github.com/docker/docker
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…#1909)

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.25.7 to 3.25.8.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@f079b84...2e230e8)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* feat(signature): Checksum signature verification

Signed-off-by: Shubham Hibare <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>

* Update message

Signed-off-by: Shubham Hibare <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>

* address comments

Signed-off-by: Shubham Hibare <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>

* consider -v flag across supported releases

Signed-off-by: Alex Goodman <[email protected]>

* add tests for install.sh signature verification

Signed-off-by: Alex Goodman <[email protected]>

* check that release is run from main

Signed-off-by: Alex Goodman <[email protected]>

* summarize install.sh flags and recommendations

Signed-off-by: Alex Goodman <[email protected]>

* remove regex use on cosign verify-blob

Co-authored-by: Dominique Martinet <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>

* simplify the compare_semver install function

Co-authored-by: Dominique Martinet <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>

* add more tests to compare_semver

Signed-off-by: Alex Goodman <[email protected]>

* nit copy change for install help

Signed-off-by: Alex Goodman <[email protected]>

* keep original compare_semver implementation

Signed-off-by: Alex Goodman <[email protected]>

* update copy to include default install path

Signed-off-by: Alex Goodman <[email protected]>

---------

Signed-off-by: Shubham Hibare <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Co-authored-by: Alex Goodman <[email protected]>
Co-authored-by: Dominique Martinet <[email protected]>
Bumps [actions/checkout](https://github.com/actions/checkout) from 4.1.1 to 4.1.6.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v4.1.1...a5ac7e5)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): update tools to latest versions

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* chore: update releaser file to compatible fields

Signed-off-by: Christopher Phillips <[email protected]>

---------

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Christopher Phillips <[email protected]>
Co-authored-by: westonsteimel <[email protected]>
Co-authored-by: Christopher Phillips <[email protected]>
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: westonsteimel <[email protected]>
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: westonsteimel <[email protected]>
…or-external-sources' into feat/allow-configuring-timeout-for-external-sources
@pouyan021
Copy link
Author

Hey @spiffcs, I rebased this branch again, could you please trigger the checks so we can hopefully merge this?

@pouyan021
Copy link
Author

cc @wagoodman

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow configurting timeout for external-sources
8 participants