Adds rattler-build recipe compatibility (and adds jolt-physics)

[baszalmstra]

WIP: This PR is a testbed to test support for rattler-build.

Checklist

• Title of this PR is meaningful: e.g. "Adding my_nifty_package", not "updated meta.yaml".
• License file is packaged (see here for an example).
• Source is from official source.
• Package does not vendor other packages. (If a package uses the source of another package, they should be separate packages or the licenses of all packages need to be packaged).
• If static libraries are linked in, the license of the static library is packaged.
• Package does not ship static libraries. If static libraries are needed, follow CFEP-18.
• Build number is 0.
• A tarball (url) rather than a repo (e.g. git_url) is used in your recipe (see here for more details).
• GitHub users listed in the maintainer section have posted a comment confirming they are willing to be listed there.
• When in trouble, please check our knowledge base documentation before pinging a team.

[conda-forge-webservices]

Hi! This is the friendly automated conda-forge-linting service.

I was trying to look for recipes to lint for you, but couldn't find any.
Please ping the 'conda-forge/core' team (using the @ notation in a comment) if you believe this is a bug.

[github-actions]

Hi! Thanks for your contribution to conda-forge.
We appreciate your effort in improving our project.
However, it looks like some changes were made outside the recipes directory.
To ensure everything runs smoothly, please make sure that recipes are only added to the recipes folder and no other files are changed.
Please make sure that any changes are reverted before you submit your pull request for review.

If these changes are intentional (and you aren't submitting a recipe), please attach a maintenance label to the PR.
Thanks!

[jaimergp]

This is going to get out of date very soon after stdlib('c').

[baszalmstra]

Is this something you want me to change?

[jaimergp]

No, no, don't worry. Just noticed it while reading, so flagged it for the curious readers. Not part of the scope of this PR!

[jaimergp]

Also I found other workflows with references to meta.yaml that you might like to adjust for nicer UX / less confusion:

staged-recipes/.github/workflows/correct_directory.yml

Lines 30 to 36 in c4f138c

	body: 'Hi! Thanks for your contribution to conda-forge.' +
	'\nUnfortunately, the recipe was added directly in the `recipes` folder without its own subfolder.\n' +
	'Please move the recipe file into a folder with the name of the package you want to submit.\n\n' +
	'For example: if your recipe is currently under `recipes/<your_package>.yaml`, ' +
	'it should be moved to `recipes/<your_package>/meta.yaml`.\n' +
	'Thanks!'
	})

This block below reminds me we might want an example file for recipe.yaml too:

staged-recipes/.github/workflows/do_not_edit_example.yml

Line 6 in c4f138c

- 'recipes/example/meta.yaml'

The "new PR template":

staged-recipes/.github/pull_request_template.md

Line 45 in d998ffa

- [ ] License file is packaged (see [here](https://github.com/conda-forge/staged-recipes/blob/5eddbd7fc9d1502169089da06c3688d9759be978/recipes/example/meta.yaml#L64-L73) for an example).

[baszalmstra]

Thanks for the early review @jaimergp ! Much appreciated!

The biggest issue I face atm is that rattler-build doesn't read the proper conda_build_config.yamls. This makes rattler-build not properly use the pinnings. (This is why the win-64 build is failing). I am contemplating either adding this to rattler-build or doing some magic in build_all.py. WDYT?

[jaimergp]

There are some elements to consider, but I'm not fully sure of how this works, so just a couple ideas:

• conda-forge-ci-setup installs conda-forge-pinning in base. The CBC from conda-forge-pinning is placed in the root of $PREFIX.
• This is then combined with the yaml files in .ci_support and, optionally, a recipe-adjacent CBC (e.g. recipes/<project>/conda_build_config.yaml).

The code is here:

staged-recipes/.ci_support/build_all.py

Lines 154 to 171 in d998ffa

	def get_config(arch, channel_urls):
	exclusive_config_files = [os.path.join(conda.base.context.context.root_prefix,
	'conda_build_config.yaml')]
	script_dir = os.path.dirname(os.path.realpath(__file__))
	# since variant builds override recipe/conda_build_config.yaml, see
	# https://github.com/conda/conda-build/blob/3.21.8/conda_build/variants.py#L175-L181
	# we need to make sure not to use variant_configs here, otherwise
	# staged-recipes PRs cannot override anything using the recipe-cbc.
	exclusive_config_file = os.path.join(script_dir, '{}.yaml'.format(
	get_config_name(arch)))
	if os.path.exists(exclusive_config_file):
	exclusive_config_files.append(exclusive_config_file)
	config = conda_build.api.Config(
	arch=arch, exclusive_config_files=exclusive_config_files,
	channel_urls=channel_urls, error_overlinking=True,
	)
	return config

So you can either replicate that logic in rattler-build, or get ahold of get_config and render a temporary YAML you can pass as an argument? Not sure what's easier, tbh 😂

[jaimergp]

I see two warnings in the macOS logs, maybe related to the SDK versions, deployment targets, etc etc.

During build:

 │ │ [139/140] Linking CXX shared library libJolt.dylib
 │ │ ld: warning: dylib ($PREFIX/lib/libc++.dylib) was built for newer macOS version (10.13) than being linked (10.9)

During analysis:

 │ │ ⚠ warning Overlinking against "/usr/lib/libSystem.B.dylib" for "lib/libJolt.dylib"

Not sure if they are related, or just noise, or whether important, but I thought I'd flag it in case there's a problem with how the CBCs are being loaded etc.

[jaimergp]

Also, there's some funky slash dance in the artifact content listing on Windows, although I guess that's rattler-build specific, not the implementation in this setup.

 │ │   - info/about.json
 │ │   - info\hash_input.json
 │ │   - info/index.json
 │ │   - info/licenses/LICENSE
 │ │   - info/paths.json
 │ │   - info/recipe/recipe.yaml
 │ │   - info/recipe/rendered_recipe.yaml
 │ │   - info/recipe/variant_config.yaml
 │ │   - info/tests\tests.yaml

[jaimergp]

This import block makes me wanna run isort. Maybe in another PR.

[baszalmstra]

I ran ruff format and isort..

[jaimergp]

This would become a problem locally if the user accidentally commits after debugging something, but I think we copy the staged-recipes folder before we start the script, so we are ok.

[jaimergp]

Suggested change

fp.close()

No need; once you exit the context, it's done for you.

[baszalmstra]

I close it because rattler-build will need to read it, but if the file is already open for writing it won't be able to open the file.

[jaimergp]

Suggested change

	with tempfile.NamedTemporaryFile(delete=False) as fp:
	with tempfile.NamedTemporaryFile(delete=False) as fp:

Maybe we should register this with atexit so it's removed once done.

[jaimergp]

I think this block should also mention the main differences across formats. Namely:

• No selectors, but if-then-else expressions (and link to docs)
• Jinja uses ${{ }}

[jaimergp]

Update this block so it refers to the new syntax for selectors.

[baszalmstra]

I added information to the top block.