feat: add shard execution workflow

[polvalente]

Adds the initial version of the process communication structure for sharded execution.

Does not handle container outputs for the sharded function yet,
and also does not yet bring everything together into the compiler jit function.

[polvalente]

I'm not sure we want to have this here, actually.
In fact, I think we might actually want to go with gen_stage for the execution, since the whole "chain of processes producing data to one another" smells a lot like gen_stage.

Thoughts?

[josevalim]

We should talk about it but I doubt GenStage will be helpful here. One of the biggest pitfalls in GenStage is that people move data around too much, when they should not. It is cheaper to move computations than to move data.

[josevalim]

Plus the whole demand approach is unnecessary here. Here is either pending or done (like a promise), no?

[josevalim]

Why do you need several providers instead of one that receives the index and returns the relevant one?

[josevalim]

Hrm, this may actually be the simplest, so nevermind.

[josevalim]

Could we fully decouple the workflow definition and execution from Nx? Ideally we would have a workflow like this:

workflow = %{
  0 => %{
    code: &foo(&1, &2, ...),
    args: [1, 2]
  },
  1 =>  %{
    code: &bar(&1),
    args: [2]
  },
  2 => %{
    code: &baz/0,
    args: []
  }
}

And then we pass this to a ProcessExecutor which is completely independent of Nx and tensors. You could also have a Nx executor, but the overall idea is that the Executor should worry about resources and not necessarily tensors (except the resources the tensors are located).