Task.run wait for inputs

[albertz]

New options:

• WAIT_PERIOD_FOR_INPUTS_AVAILABLE
• TASK_INPUTS_MUST_BE_AVAILABLE

Fix #159.

Note that TASK_INPUTS_MUST_BE_AVAILABLE is True by default, which can potentially break some old setups, where jobs used an input which was not an existing file but only the prefix of it (e.g. checkpoint file name).

[michelwi]

Note that TASK_INPUTS_MUST_BE_AVAILABLE is True by default, which can potentially break some old setups, where jobs used an input which was not an existing file but only the prefix of it (e.g. checkpoint file name).

I personally am fine with the new default behavior. But I would like to check with @patrick-wilken if the MT recipes make active use of this "feature". If yes, then we should rather have FALSE the default.

[patrick-wilken]

As said on Slack, I would be okay with having to set TASK_INPUTS_MUST_BE_AVAILABLE to make our current recipes work. WAIT_PERIOD_FOR_INPUTS_AVAILABLE=0 would then also make sense until we stop using path prefixes, right?

[michelwi]

As said on Slack, I would be okay with having to set TASK_INPUTS_MUST_BE_AVAILABLE to make our current recipes work.

ok.

WAIT_PERIOD_FOR_INPUTS_AVAILABLE=0 would then also make sense until we stop using path prefixes, right?

yes, unless you want to wait unnecessarily for each expected-missing input.