Create a documented outline of the 3rd party Reproducible Attestation process

[andrew-m-leonard]

Process Outline:

At Adoptium:

1. Temurin release build pipeline runs a new post-build job ("Verify Reproducible") step that verifies the success of the Rebuild_Same_JDK_Reproducibility_Test test

• The Rebuild_Same_JDK_Reproducibility_Test reproducible nn% is then recorded by the "Verify Reproducible" job and creates a CycloneDX CDXA document with "evidence" from the test, and adds it to the build job artifacts for that platform, and updates the SBOM to reference via a bom-link the new CDXA BOM

2. The Temurin release is published to the temurinNN-binaries along with the SBOM + CDXA
3. The Temurin binary in the Marketplace is marked a "Reproducible status - nn% - Pending 3rd party verification"

A 3rd Party:

1. A "3rd party" performs an independent "Reproducible Verification Build" (https://adoptium.net/blog/2024/08/adoptium-reproducible-verification-builds/) of the same published Eclipse Temurin release
2. Assuming 100% successful, they record the necessary "evidence" in order to attest to the reproducibility of the Eclipse Temurin binary
3. Using a CDXA json file template (supplied by Eclipse Adoptium), they record the required attestion and evidence
4. The CDXA json file is "Signed" by the 3rd party Certificate. (mechanism??, ideally JSF using a PEM, but that may not be easy for 3rd party...)
5. The 3rd party uses the Adoptium Marketplace to place the Signed CDXA document into the marketplace folder. (Q: This process implies the 3rd party must be an Adoptium Marketplace Member??)
6. The receipt of the CDXA document by the Adoptium Marketplace triggers an automated "CDXA Verify Policy", which runs a script to validate the content and signature of the CDXA against the Adoptium binary and Adoptium CDXA.
7. If the "CDXA Verify Policy" is successful, then:

• The Temurin artifact metadata is updated to record reproducibleVerified=true
• The Temurin SBOM is updated with a bom-link to the purl of the 3rd party CDXA
• The Adoptium Marketplace build entry gets a "Reproducible Verified" tick !

[andrew-m-leonard]

@tellison @jiekang @mbarbero Can I get your comments on above please? thanks

[michaelwinser]

I would change "Un-verified" to "Pending 3rd party verification" or something to that effect. Bonus, a link to "instructions on how to verify" so that any 3rd party could reproduce the build and provide an attestation would be interesting. Handwave.

Assuming that "%age" is short for percentage, I'm not sure I understand it in this context.

I actually like that Step 5 requires that the verifier be a marketplace member unless the barrier to entry is too high. It changes the "someone verified this" to "someone known to us with some kind of contractual relationship verified this" or even better "Company X verified this." That means a lot to the consumer.

I think the reproducibleVerified attribute will need to be a counter and we will eventually want some way to get the list of verifiers.

We had discussed having a specific release for "reproducible" so that someone could query for builds that were reproducible. That might not be necessary but I do like the idea of a clear release type.

[andrew-m-leonard]

I would change "Un-verified" to "Pending 3rd party verification" or something to that effect. Bonus, a link to "instructions on how to verify" so that any 3rd party could reproduce the build and provide an attestation would be interesting. Handwave.

Yep, changed.

I actually like that Step 5 requires that the verifier be a marketplace member unless the barrier to entry is too high. It changes the "someone verified this" to "someone known to us with some kind of contractual relationship verified this" or even better "Company X verified this." That means a lot to the consumer.

Agree

I think the reproducibleVerified attribute will need to be a counter and we will eventually want some way to get the list of verifiers.

Yes, that's probably what we will need

We had discussed having a specific release for "reproducible" so that someone could query for builds that were reproducible. That might not be necessary but I do like the idea of a clear release type.

So the Adoptium API will be extended to return something like "latest&verified", which will return the latest artifact with reproducibleVerified>0, as opposed to "latest" which will return simply the latest as today.

[mbarbero]

Do you envision this as a method for third parties to verify Adoptium's builds only? This would imply that on a page like this:

no vendor other than Adoptium would receive the "verified" tick. Is that correct?

[smlambert]

no vendor other than Adoptium would receive the "verified" tick. Is that correct?

true at this moment in time, but not necessarily in the future, since changes to support reproducibility were pushed upstream, to allow other distributors to also benefit

[tellison]

I'd like to separate out the basic mechanism of recording verification build attestations from the details of a visual depiction of a verification (tick marks etc), and the policy of who we recognise as a trusted verifier (working group members etc). Though I do strongly believe that the mechanism we pick should allow anyone to record a verification attestation.

I propose:
(a) we store reproducibility attestations in a public immutable ledger (such as Sigstore Rekor) designed for this task, preferably hosted by the Eclipse Foundation itself.
(b) when Adoptium produces a Temurin binary and its associated SBOM, the SBOM contains a fixed attestation ledger URL.

To record reproducible attestations

1. Anyone (including Adoptium) can use the SBOM to reproduce the Temurin binary, and record that fact in metadata. This is suggested above to be CDXA format.
2. Anyone can store their attestation metadata in the immutable ledger.

To query reproducible attestations

1. The SBOM for a build includes the attestation ledger URL. All attestations can be retrieved by querying the ledger.
2. Adoptium will produce a new api.adoptium.net path to query the ledger and return the attestations for a build.

Reproducible attestations depictions

1. The website can be updated to use the Adoptium API and depict attestations/verifications of binaries in a visually pleasing manner.
2. The project can decide to make some attestations more prominent (e.g. Working Group members), etc.

p.s. Although I reference Sigstore above, I do note that they don't list CycloneDX Attestations as a currently supported type - potentially requiring a new definition. The store does support Intoto attestations today, but staying with CDX makes sense.

[andrew-m-leonard]

Thanks @tellison I like the idea of using a "Rekor" ledger.

There's a few technical issues/questions with Rekor:

• Ideally using an Eclipse Founderation Rekor instance would be ideal. Does EF have a Sigstore Rekor instance already @mbarbero @netomi ?
  • If EF were to create an instance, it would need to be writable by any 3rd party, would that have any implications to EF? possibly from a security standpoint?
• If we were to start initially using the public sigstore Rekor instance (https://rekor.sigstore.dev/) and then maybe move to a EF instance, we would probably need a "migration" or "compatibility" of both from the API..?

[mbarbero]

Ideally using an Eclipse Founderation Rekor instance would be ideal. Does EF have a Sigstore Rekor instance already @mbarbero @netomi ?

No, we don't host our own Rekor instance, nor we plan to host one currently.

[andrew-m-leonard]

2. Anyone can store their attestation metadata in the immutable ledger.

Having researched Rekor, it is actually not possible to "store" anything in Rekor, as it is not a "Content Store", it is purely a Signature Ledger for which you must know one of the "email of signer"/"Hash"/"SHA"/UUID/LogIndex of. In other words in your System of interest for which you are enforcing some security policies, whenever you "attest" to something (Fred compiled X.c) you sign the "ledger" and record a reference in your system to that signed "ledger" entry.

To query reproducible attestations

1. The SBOM for a build includes the attestation ledger URL. All attestations can be retrieved by querying the ledger.

As per above, this is only possible if the SBOM contains the ledger entry URL by UUID entry.

2. Adoptium will produce a new `api.adoptium.net` path to query the ledger and return the attestations for a build.

As per above, we will need to "store" the ledger entry UUIDs "somewhere", so whenever someone "attests" then that new ledger UUID needs storing in the Adoptium DB for return by this new API. As it stands the only "sort of writeable" access to this data is as an Adoptium Member Vendor, who could store the Ledger UUID entry in their MarketPlace Data.

[andrew-m-leonard]

For the first pass, I propose we don't use a Rekor ledger, as it does not provide much use beyond what is already available as an Adoptium marketplace Vendor.
To properly leverage Rekor, we will need a more "public" writeable entry to the Adoptium MarketPlace.

For the first phase we will just use CycloneDX CDXA documents within the Adoptium Marketplace data.

[andrew-m-leonard]

Useful reference: https://stacklok.com/blog/decoding-rekor-understanding-sigstores-transparency-log