Difficulty with passing environment variables between Jinja2 and Cylc 8 config file sections

[sk8forether]

Description of Problem

It turns out that in Cylc 8 it is non-trivial to pass certain types of variables around within the global, workflow, or task configuration, even within the same file. And it becomes even more difficult to do this within conditional statements, user environment variables, or cylc runtime environment variables.

The Cylc 8 docs are littered with examples of using Jinja2 conditional statements to do all kinds of nifty things within the workflow, but these conditional statements have harsh limitations about which types of variables you can use. Cylc's implementation of Jinja2 also limits which features are available in the global configuration, thus causing one to not be able to do much passing back and forth without the configuration failing. This issue is mostly to document a bunch of testing that was done for the purposes of using these conditional statements within the global config for various purposes, including customization of runtime configurations based on platform selection (will be covered more in another issue).

Testing

(TODO: Will add more details here)

References

From the docs

• Jinja2 in Cylc 8 docs
• Global Configuration
• Task Parameters and Task Configuration have examples of shell variables used in configs
• Job Script Variables defined at runtime by Cylc are not available during configuration check of the global config, at least. See also Job Script Variables section in Task Configuration

Stack Overflow Questions

Environment Variables

Note that all of these methods were tested and failed due to Cylc's specialized implementation of Jinja2

• Accessing OS environment variables from Jinja2 template
• Get or Check Current Environment in Jinja2
• How to get an env var inside a jinja2 template in ansible
• Render a environment variable in jinja template
• how to set the environment variables in python flask and jinja2

Logic operators

• How to use logic operators in jinja template
• Logical operators in jinja template
• Jinja2 shorthand conditional
• How to use conditional if statements in Jinja 2?
• Jinja2 - how to assign value to a variable based on if else condition?

Other

• Jinja2: Check If Variable – Empty | Exists | Defined | True
• Basic Syntax of Jinja
• Jinja2 Tutorial - Part 2 - Loops And Conditionals