Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs-only and source-only archives #50

Open
sdarwin opened this issue Oct 23, 2023 · 1 comment · May be fixed by #52
Open

docs-only and source-only archives #50

sdarwin opened this issue Oct 23, 2023 · 1 comment · May be fixed by #52

Comments

@sdarwin
Copy link
Contributor

sdarwin commented Oct 23, 2023

@alandefreitas you had mentioned at some point the idea of creating docs-only and source-only archives for boost.

could you add that in release-tools?

@glenfe just informed us the jfrog downloads are taking 200TB of bandwidth per month.

Switching to source-only downloads would save $1000's per month.

These are the hosted bundles https://boostorg.jfrog.io/artifactory/main/

beta/     07-Apr-2021 22:52    -
develop/  07-Apr-2021 22:48    -
master/   07-Apr-2021 22:44    -
release/  07-Apr-2021 20:44    -
staging/  04-Aug-2023 13:51    -

There are two places in release-tools where docs-only and source-only archives might be generated.

  1. https://github.com/boostorg/release-tools/blob/develop/ci_boost_release.py
  2. https://github.com/boostorg/release-tools/blob/develop/publish_release.py

There are pros and cons to having the logic in one or the other. Most of the complexity has usually been contained in ci_boost_release.py so that is the likely place. Create subdirectores in jfrog under master/ and develop/.

develop/boost_1_83_0-snapshot.tar.gz   (keep the same for now)
develop/full/boost_1_83_0-snapshot.tar.gz   
develop/docs/boost_1_83_0-docs-snapshot.tar.gz   
develop/source/boost_1_83_0-snapshot.tar.gz   

master/boost_1_83_0-snapshot.tar.gz    (keep the same for now)
master/full/boost_1_83_0-snapshot.tar.gz   
master/docs/boost_1_83_0-docs-snapshot.tar.gz   
master/source/boost_1_83_0-snapshot.tar.gz   

Arguably, the name of the snapshots should always stay the same, such as boost_1_83_0-snapshot.tar.gz , and the determining factor is the directory where it is hosted. This makes it easier in other scripts which consume the archives, to simply switch the location (from full/ to source/) .

Possibly, the docs-only archive could have a -docs added to the name.

ci_boost_release.py generates and uploads the various snapshot to jfrog.

publish_release.py at the time of a release, downloads snapshots, renames them to release/1.83.0/source/boost_1_83_0.tar.gz, and uploads again.

release/1.83.0/source/
release/1.83.0/docs/
release/1.83.0/full/

Unfortunately, the current archives are using the name "source" already.

I was hoping the first iteration of this feature would publish to multiple folders, but somehow keep the current behavior for a short time, since the old website depends on /source/ actually being /full/.

Perhaps temporarily then:

release/1.83.0/source-only/
release/1.83.0/source/   (actually full)
release/1.83.0/docs/
release/1.83.0/full/
@alandefreitas
Copy link
Member

Sure. I just have to merge the code from cppalliance/boost and review it, as we did for the antora functionality.

That's good news. I didn't know this change was also that important. I actually thought there would be a lot of debate over it.

The defaults you suggested are already similar to the names I used in cppalliance/boost so that should be fine.

@alandefreitas alandefreitas linked a pull request Oct 26, 2023 that will close this issue
alandefreitas added a commit to alandefreitas/release-tools that referenced this issue Oct 26, 2023
Add extra archive variants such as docs-only and source-only. These variants can reduce expenses with JFrog download bandwidth, provide users with archives that are simpler to use, and provide docs-only archives for the website.

The MakeBoostDistro.py script includes parameters to determine what types of files should be included in the distribution. All other functions are adapted to handle these requirements accordingly.

fix boostorg#50
alandefreitas added a commit to alandefreitas/release-tools that referenced this issue Oct 26, 2023
Add extra archive variants such as docs-only and source-only. These variants can reduce expenses with JFrog download bandwidth, provide users with archives that are simpler to use, and provide docs-only archives for the website.

The MakeBoostDistro.py script includes parameters to determine what types of files should be included in the distribution. All other functions are adapted to handle these requirements accordingly.

fix boostorg#50
alandefreitas added a commit to alandefreitas/release-tools that referenced this issue Oct 27, 2023
Add extra archive variants such as docs-only and source-only. These variants can reduce expenses with JFrog download bandwidth, provide users with archives that are simpler to use, and provide docs-only archives for the website.

The MakeBoostDistro.py script includes parameters to determine what types of files should be included in the distribution. All other functions are adapted to handle these requirements accordingly.

fix boostorg#50
alandefreitas added a commit to alandefreitas/release-tools that referenced this issue Oct 27, 2023
Add extra archive variants such as docs-only and source-only. These variants can reduce expenses with JFrog download bandwidth, provide users with archives that are simpler to use, and provide docs-only archives for the website.

The MakeBoostDistro.py script includes parameters to determine what types of files should be included in the distribution. All other functions are adapted to handle these requirements accordingly.

fix boostorg#50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants