Generic Workflow Pipeline

[markmcdowall]

A use should be able to dynamically define a workflow based on available tools and then hooking up the inputs and outputs together.

In the refactor_module branch there is initial work being done to move the Workflow classes into mg_process_fastq/workflow. There is also a new pipeline called process_pipeline.py that has 2 extra input parameters. The first is --workflow, this is the module name (mg_process_fastq.workflow.biobambabm) and --pipeline which is the name of the class. This makes it possible for all workflows to be run using a single script from the command line rather than having a new script for each workflow.

To get further than this there is the need for defining an execution path for representation within the JSON files and then implementing the logic within the process-pipeline.py file. It would also be good to move the process-pipeline.py script to the mg-tool-api repo so that it is a core script that is installed as default.