Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Full circle: Nextflow #93

Merged
merged 16 commits into from
May 30, 2024
Merged

Full circle: Nextflow #93

merged 16 commits into from
May 30, 2024

Conversation

jmfernandez
Copy link
Member

This merge includes many fixes around RO-Crate generation and consumption, as well as processing hints.

  • Generated RO-Crates with detached inputs were not properly including either contentUrl or identifier properties.
  • SPARQL queries were not capturing the optional nature of input and output contentUrl , and added querying about identifier as an alternate for contentUrl.
  • Code now considers identifier as an alternate provider of file or dataset input PIDs.
  • The concept of synthetic output has been added, in order to distinguish output parameters from synthetic ones added to ease the selection of results from an output directory.
  • Hints about output path linking to parameters (needed for Nextflow and future Snakemake support) have been added.
  • Hints about glob patterns for synthetic outputs have also been added.

…e really outputs.

Also, add `valueRequired` property to output FormalParameters.
Also, slightly change on root dataset query to get its `isBasedOn`, as WorkflowHub is now automatically setting it.
…`, calling methods from `wfexs_backend.utils.rocrate` toolbox.
…language has a clear separation between inputs and outputs.

Also, much better "expected" and "materialized" output parameter handling.

Last, but not the least important, now each run has its own separate intermediate directory, to completely isolate runs.
Now, each autofilled input injects an autofilled expected output labelled as not synthetic, in order to distinguish real (i.e. parameterized) from synthetic outputs (i.e. based on glob patterns or output directory exploding).
The scenario being tested is one using a Nextflow workflow, where
output parameters are not explicitly declared by nextflow itself. So,
several custom terms have to be used to properly describe the execution.

Also, some corner cases have been caught.
… were not including it in either `contentUrl` or `identifier`.
… could come from `identifier` instead of `contentUrl`.

Basic Nextflow workflows are now properly imported!!!!
@jmfernandez jmfernandez merged commit 39f06d0 into main May 30, 2024
7 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant