providing instructions to NCO about installing libraries and utilities that we request for our modeling systems

[edwardhartnett]

@aerorahul brought up this in email:

[...]How we are providing instructions to NCO about installing libraries and utilities that we request for our modeling systems:

We are relying on hpc-stack branch nco-wcoss2 to provide them instructions. For some libraries only that don't have spack recipes. NCO/GDIT follows the spack method for those that have the recipes in the authoritative spack repo.

The basic instructions for every library install or update of version, they follow this:

git clone https://github.com/NOAA-EMC/hpc-stack.git nco_wcoss2
cd nco_wcoss2
git checkout nco-wcoss2
(determine where you want to install it. It is to give the path to <prefix>. Assume prefix=/apps/ops/${envir}/libs     ${envir}=prod)
mkdir -p <prefix>
cd nco
sh onetimesetup.sh
./buildhpcstack.sh prod intel package_name_and_ver

So, for every library, they treat hpc-stack as a new, independent clone and follow the instructions. We create the relevant package_name_and_ver.yaml file in stacks/ folder and updates the libs/build_package_name.sh if there are any updates. He also updates the relevant module file in this branch.

You can see the requests we have sent this far in this folder.

Doing so, this essentially eliminates the need for a framework that was hpc-stack, and one could basically pull out the build_package_name.sh and hard-wire the dependencies and compile instructions in it for WCOSS2. The process would be much simpler, streamline and explicit.

We could just give them the build_package_name.sh and be done with it.

In making the many changes to the branch nco-wcoss2, the branch nco-wcoss2 has significantly diverged from develop. It no longer has the bugfixes that have gone into develop, let alone any development.

I am not sure how this will play out when the NCEPLIBS team rolls out spack-stack for use w/ NCO and WCOSS2.

[edwardhartnett]

@aerorahul are you referring to libraries/packages that are not in spack?

My first thought would be to create spack builds for whatever is missing and submit them to the main spack repo.

In the past they have accepted new spack builds within hours or days. Kyle and I did at least a dozen between us when we started with spack - its not hard. Mostly they take 15 or 20 minutes.

Would that eliminate the problem? If NCO could install from spack would they do that?

@Hang-Lei-NOAA and @AlexanderRichert-NOAA what do you think?

[climbfuji]

I had a chat with Rahul in Slack yesterday, let's talk about this at today's meeting if that's ok?