Consolidate Hosted OSI Documentation Sources

[TimmRuppert]

Describe the bug

Currently, there are three separate sources for OSI documentation hosted online:

• The main Antora documentation at: opensimulationinterface.github.io/osi-antora-generator
• The OSI v3.5.0 Antora documentation at: opensimulationinterface.github.io/osi-documentation/
• The Doxygen documentation at: https://opensimulationinterface.github.io/open-simulation-interface/

Describe how to reproduce the bug

1. Visit the links mentioned above.
2. Observe that the documentation is split across three sources.

Describe the expected behavior

A unified and streamlined documentation experience. Specifically, the following changes are recommended:

• Remove the dedicated hosting for the OSI v3.5.0 documentation.
• Redirect opensimulationinterface.github.io/osi-documentation to point to the latest version hosted at opensimulationinterface.github.io/osi-antora-generator.
• Since the full content of the Doxygen documentation is already included in the Antora-generated documentation, there is no need for a separate Doxygen-hosted version. Therefore, the standalone Doxygen documentation should be removed.

Show some screenshots

If applicable, add screenshots to help explain your problem.

[jdsika]

What kind of approval do you need to do as you proposed?

[TimmRuppert]

I am not entirely sure as I have not used github pages or antora yet. Furthermore, I am not familiar with the idea behind the release chains and the OSI repo split. But I think I understand the issue now a little bit better and want to summarize it for clarification:

• The opensimulationinterface.github.io subdomain part comes from the github organization
• The the url suffix paths ../osi-documentation, /osi-antora-generator and /osi-documentation simply comes from the three involved OSI github projects
• The pages are stored in these three projects on either a gh-pages branch or as an artifact (page source would then be a gh action) of each project
• The issue with the old 3.5.0 version is that it is based on the old disabled but existent asciidoctor job specified here which was executed at most a year ago
• My guess is that the 3.5.0 version is a leftover from a transition from Asciidoctor to Antora
• There is a job in this open-simulation-interface repo which deploys the doxygen documentation on pushes to master

My naive approach would be:

• In the open-simulation-interface repo (that one here)

  • Remove the job which uploads the doxygen documentation
  • Remove all files on the gh-pages branch and add a single index.html which forwards everyone to the one place where the documentation is hosted (see below)

• In the osi-documentation repo

  • Delete the asciidoc-build.yml workflow
  • Remove all files on the gh-pages branch and add a single index.html which forwards everyone to the one place where the documentation is hosted (see below)
  • Update the information about the documentation in this page of the documentation itself.

• Leave the documentation in the osi-antora-generator repo where it already is and use that as a single source

In that case I would have enough rights to simply create PRs.

But this is just my naive way of looking at the problem, e.g. there is also the ASAM Antora Extensions responsible for the doxygen conversion etc. Most of the documentation setup seems to come from @pmai and @philipwindecker. Maybe one of them can have a look at my proposal and point out potential flaws or pitfall.

One issue is that (external) links to the osi-documentation (wrong version anyways) page or the doxygen page would break. E.g. the esmini readme is not pointing to the Antora version. But I think this is still acceptable as it would unify the documentation to a single source.

[jdsika]

@engelben is there still support from the office regarding these setup of documentation etc?

[philipwindecker]

Hi @TimmRuppert , @jdsika

I had to check with the ASAM office first, but we cleared everything and I can now support you on this.
At first glance, the proposed solutions look good. I will just have to check a few things (e.g. if the doxygen job was used for anything else that would still be relevant), but I am quite positive that there should be no hidden hiccups.

I will check everything and get back to you here with my insights.

[philipwindecker]

Ok, I went through the repositories and tried to check everything by eye. I would say that the suggested approach works, some small additions might be useful though.

List of correct links

Type	Latest	v3.5.0
Specification	https://opensimulationinterface.github.io/osi-antora-generator/asamosi/latest/specification/index.html	https://opensimulationinterface.github.io/osi-antora-generator/asamosi/V3.5.0/specification/index.html
Doxygen	https://opensimulationinterface.github.io/osi-antora-generator/asamosi/latest/gen/index.html	https://opensimulationinterface.github.io/osi-antora-generator/asamosi/V3.5.0/gen/index.html
OSMP	https://opensimulationinterface.github.io/osi-antora-generator/asamosi/latest/sensor-model/osi-sensor-model-packaging_spec.html	https://opensimulationinterface.github.io/osi-antora-generator/asamosi/V3.5.0/sensor-model/osi-sensor-model-packaging_spec.html
DCO	https://opensimulationinterface.github.io/osi-antora-generator/asamosi/latest/specification/contributing/dco.html	https://opensimulationinterface.github.io/osi-antora-generator/asamosi/V3.5.0/specification/contributing/dco.html

Redirect page content

<!DOCTYPE html>
<meta charset="utf-8">
<meta http-equiv="refresh" content="0; url=https://opensimulationinterface.github.io/osi-antora-generator/asamosi/latest/specification/index.html">
<meta name="robots" content="noindex">
<title>Redirect Notice</title>
<h1>Redirect Notice</h1>
<p>The page you requested has been relocated to <a href="https://opensimulationinterface.github.io/osi-antora-generator/asamosi/latest/specification/index.html">https://opensimulationinterface.github.io/osi-antora-generator/asamosi/latest/specification/index.html</a>.</p>

open-simulation-interface

• Remove steps Deploy to gh-pages if push to master branch from

  open-simulation-interface/.github/workflows/protobuf.yml

  Line 140 in b00ad2b

  - name: Deploy to gh-pages if push to master branch

  and Archive Documentation from

  open-simulation-interface/.github/workflows/protobuf.yml

  Line 125 in b00ad2b

  - name: Archive Documentation

• Don't know if this is generated or not, but https://github.com/OpenSimulationInterface/open-simulation-interface/blob/master/osi_version.proto.in contains a link to the osi-documentation gh-pages as well (should be changed if used)
• Replace https://opensimulationinterface.github.io/osi-documentation/open-simulation-interface/doc/howtocontribute.html#developer-certification-of-origin-dco with https://opensimulationinterface.github.io/osi-antora-generator/asamosi/latest/specification/contributing/dco.html in https://github.com/OpenSimulationInterface/open-simulation-interface/blob/master/.github/pull_request_template.md
• Replace all content on branch https://github.com/OpenSimulationInterface/open-simulation-interface/tree/gh-pages with a single file

osi-documentation

• The workflow https://github.com/OpenSimulationInterface/osi-documentation/blob/master/.github/workflows/asciidoc-build.yml has been deactivated anyway, but since it is not used any longer, it can also be deleted for good
• https://github.com/OpenSimulationInterface/osi-documentation/blob/master/README.adoc contains links for gh-pages not on the generator repository. These should be replaced
• https://github.com/OpenSimulationInterface/osi-documentation/blob/master/content/general_docs/osi_repos.adoc contains links to gh-pages not on the generator repository. These should be replaced
• Replace https://opensimulationinterface.github.io/osi-documentation/open-simulation-interface/doc/howtocontribute.html#developer-certification-of-origin-dco with https://opensimulationinterface.github.io/osi-antora-generator/asamosi/latest/specification/contributing/dco.html in https://github.com/OpenSimulationInterface/osi-documentation/blob/master/.github/pull_request_template.md
• Replace all content on branch https://github.com/OpenSimulationInterface/open-simulation-interface/tree/gh-pages with a single file

osi-sensor-model-packaging

• Replace https://opensimulationinterface.github.io/osi-documentation/open-simulation-interface/doc/howtocontribute.html#developer-certification-of-origin-dco with https://opensimulationinterface.github.io/osi-antora-generator/asamosi/latest/specification/contributing/dco.html in https://github.com/OpenSimulationInterface/osi-sensor-model-packaging/blob/master/.github/pull_request_template.md

[philipwindecker]

Side notes:

GitHub Pages is quite simple for us. We just have a specific action run that pushes content to a given branch ("gh-pages") and then host that based on what it finds.
This, it may work if we just replace the index.html`s content in each of these branches (and remove the rest as it is not needed any longer). If this does not work, we could run a simple once-and-done pipeline that does this with an action (basically the same as the current job, just using different input).

Regarding the extensions: That topic is a bit more complicated. The repository with them is available on ASAM GitLab, so if you have access there, you can also get to that repository (the url can be found in the submodules of the generator).
However, it is a lot of code to be digested, so the interesting information here should be that the doxygen extension I wrote basically runs doxygen for each version of the specification where the required configuration is found (currently, everything from 3.5.0 forward). This generated doxygen output is then converted into a format that Antora (and, by extension, Asciidoctor) understands and added as virtual files similar to those that Antora creates in the background when generating the Antora output.
The original doxygen job is therefore not needed for any Antora output. One major reason for this approach was that the content of each version may differ from the previous or the next, meaning that also the doxygen output would be different. The original approach with a dedicated doxygen job, however, only creates the output for the latest version and would thus not allow access to previous definitions if needed. Since Antora already works with versions, we decided to create the Doxygen extension that runs for each version separately.

In case there is more interest in this, I could also offer a deep dive into both the (release) pipelines and the Antora extensions. Just let me know. :)

[philipwindecker]

@TimmRuppert would you like to apply these changes yourself or would you rather have me do them?
In any case, I can offer to review your changes (or involve you in the review).

I am, however, a bit tight on time this week, so I may only get to it next week or so.

P.S.: I also noted that both release candidates are still included and, by naming convention from Antora, the rc-2 is considered "latest" (not the released version).

Related info on this: https://docs.antora.org/antora/latest/how-component-versions-are-sorted/#version-schemes

[jdsika]

I would be happy if you could do it @philipwindecker next week, thank you very much!

[philipwindecker]

Alright, will do. I'll let you know when I have everything prepared and ready for review.

[TimmRuppert]

Thank you very much for the side not and the summery!

@TimmRuppert would you like to apply these changes yourself or would you rather have me do them?
In any case, I can offer to review your changes (or involve you in the review).

I would be happy if you could do it @philipwindecker next week, thank you very much!

Even though @jdsika has already asked you to do it, I would also have preferred if you had done it: You are clearly the expert, and you found even more places that need changes.

P.S.: I also noted that both release candidates are still included and, by naming convention from Antora, the rc-2 is considered "latest" (not the released version).

This happens to us all the time and it is a little bit annoying, is there any chance we can address (in a separate issue/PR) as well? Seems like the is a prerelease key but I have basically no idea how Antora works.

[philipwindecker]

Yes, you can explicitely exclude certain versions from the build (in the site.yml in the generator). That way, the tag can persist without it being included in future builds.