[WIP] Decouple examples gallery from the main repository #251
Conversation
Codecov Report
@@ Coverage Diff @@
## master #251 +/- ##
=======================================
Coverage 80.78% 80.78%
=======================================
Files 140 140
Lines 10625 10625
=======================================
Hits 8583 8583
Misses 2042 2042
Flags with carried forward coverage won't be shown. Click here to find out more. Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Fantastic job @aperezhortal!
I only have few understanding questions and there are a couple of super minor changes that we need to include (basically updating the references to the gallery page).
.github/workflows/execute_tutorial_notebooks_in_external_repo.yml
Outdated
Show resolved
Hide resolved
| # Trigger the notebooks' execution in the external repository. | ||
| # | ||
| # If the "main" branch triggered this event, then the nb execution is triggered in the | ||
| # "main" branch, otherwise in the dev branch. |
There was a problem hiding this comment.
I really like this idea! So, did I understand correctly that all branches but master will automatically render into "dev"?
There was a problem hiding this comment.
Yes. However, right now only the main branch is pull in the examples repo. I will add something to pull the same branch that sent the triggering commit to the external repo. In that way, we can test if the examples execute correctly with development branches.
.github/workflows/execute_tutorial_notebooks_in_external_repo.yml
Outdated
Show resolved
Hide resolved
|
I also noticed that |
Just a consequence of the branch that I was using 😄 |
|
I have another annoying comment (sorry): it should be "Example gallery" all over instead of "Examples gallery", shouldn't it? |
Yes, you are right. I will fix it shortly. I will also change the PR to Work In progress since I'm experimenting with a slightly different workflow where the rendering is triggered by empty commit messages to a dedicated branch. This allows finer control on which pysteps branch to pull, which branch in the gallery to render, and what RTD branch to trigger. 🤓 |
aperezhortal
changed the title
|
Yes, I agree, let's close this PR in favor of #262. |
Problem
Currently, the examples gallery is a part of the pysteps project and the examples are executed during the documentation build in Read The Docs (RTD). Running these examples is a time-consuming operation that puts us close to the RTD's build limits.
Proposed solution
The discussions in #242 converge on decoupling the example gallery from the main repo, keeping the examples in a separate repository that uses Github Actions to execute the examples and build the examples gallery. The rendered gallery is
pulledfrom the separate repo when the pysteps documentation is built in RTD.Changes introduced in this PR
examples/*.pyfiles) from the pysteps repo. The examples were moved to the pysteps_tutorials repository. Thepysteps_tutorialsrepository contains only the Sphinx documentation for the example gallery. Here, the GitHub actions render the example gallery (using sphinx-gallery) and save the rendered gallery (rst and png files) in separate branches. IMPORTANT, the examples have the form of python scripts for now. Although this can be updated to jupyter notebooks if needed.pysteps_tutorialsrepository.Discussion
Should we use Jupyter notebooks or python scripts for the example gallery?
I personally prefer python scripts. Although jupyter notebooks are good for exploratory analysis and can be nicely visualized in Github, keeping track of the changes is very difficult since the diffs are not very human-readable. On the other hand, keeping track of the differences in python scripts is straightforward. Another advantage of using python scripts is that we can benefit from the static code analysis of the IDEs.
Also,
sphinx-galleryautomatically converts the python scripts to jupyter notebooks and makes them available in the documentation. There is an option (experimental) to add theopen in binderbutton that we may explore.I check how the example galleries are handled in other projects and some use jupyter notebooks while others use scripts:
It seems that it boils down to a matter of personal preference. Both approaches are good, and if we choose to use notebooks, updating the current PR is straightforward.
What are your thoughts?