Add script to regenerate catalog_upload.json files

[ckadner]

It is becoming difficult to maintain the catalog upload files for the Bootstrapper and the Quickstart with Docker Compose.

This PR adds a helper script to regenerate the catalog_upload.json file(s) from the YAML files that are found inside the cloned katalog repo.

$ python3 hack/regenerate_catalog_upload_json.py 

Regenerating catalog_upload.json files:
 - bootstrapper/catalog_upload.json
 - quickstart/catalog_upload.json

Done. Use git diff to evaluate if and which changes are desired!

Current caveats:

• no documentation yet
• no command parameters, no command line help
• the user must have cloned the katalog repo and the mlx repo in the same parent folder (in the future those paths should be configurable as parameters)
• not all of the YAML files in the katalog repo may want to be included in the catalog upload, so after the files get regenerated, the user (MLX maintainer) needs to excercise some judgment
• assets that live outside the katalog repo will not be found and have to be re-added (or not removed) using a Git enabled IDE or git diff tool
• this initial commit reorders some of the assets (based on filename)

Verify all URLs are valid:

grep -o -h -E "http.*yaml" */catalog_upload.json | \
  while read url; do \
    curl -s -o /dev/null -w "%{http_code}" $url; echo ": ${url}"; \
  done | grep --color "404"

Signed-off-by: Christian Kadner [email protected]

[mlx-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ckadner

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [ckadner]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ckadner]

@Tomcli -- this PR adds the new katalog assets for CodeNet, Qiskit and MAX models.

It will also help to take care of future katalog updates with a new script.

[Tomcli]

/lgtm

[ckadner]

Thanks @Tomcli

[ckadner]

I will address (some of) the current caveats listed above in a separate PR: #222

• move script from /hack to /tools/python
• add developer documentation
• clone the katalog repo into a temp directory instead of relying on the user to have done it prior to running the script
• not all of the YAML files in the katalog repo may want to be included in the catalog upload, so alert the user to all the changes instead of relying on git diff
• read the catalog_upload.json file prior to regenerating it, in order to capture assets that live outside the katalog repo and re-add them later