Check yt save() Function

[cindytsai]

Check yt Save Function

Description

In inline script, we need save() outside the if yt.is_root() clause, because annotate_cquiver (and other annotations) makes save does data IO. When doing data IO in libyt (using function inside io.py), each MPI rank must call the same method. See:

• Support Getting Nonlocal Data #31
• Ask Non-Local Grid From Other MPI Rank #26

import yt
yt.enable_parallelism()
def yt_inline():
    ds = yt.frontends.libyt.libytDataset()
    slc = yt.OffAxisSlicePlot(ds, [1, 1, 0], [("gas", "density")], center="c")
    slc.annotate_cquiver(("gas", "cutting_plane_velocity_x"), ("gas", "cutting_plane_velocity_y"), factor=10, plot_args={"color":"orange"}, )
    slc.save()

Sometimes, there will be some missing figure in the output series of figures. This may happen if each rank is writing and creating a file with identical name. (link)

[cindytsai]

Possible Solution

Is it possible to move save() function inside if yt.is_root() by changing the code in yt.

Check

• Check if annotations call io.py for empty list when saving figures.
  • No, the chunk list is not empty.
• Check if the return cell number after io.py in save function is empty.
  • No, it is not empty. (This line)

[cindytsai]

TODO

• Check how and when save() calls libytIO. And also, how does it know at what operation should it calls libytIO.
• Check ParallelAnalysisInterface class.

Idea

[cindytsai]

Possible Solutions

Option 1: Every operation should be put outside of is_root and only_on_root.

This is the easiest way, but we will need to give up its performance. We do this by making the write-to-file part in save only works on root rank. Namely, only root rank can write to disk, and the other rank does the redundant operation except libyt dataIO.

Option 2: One can put operation like save inside is_root and only_on_root, but instead of returning None on non-root rank, we make the other rank wait at libyt dataIO.

This is the original method, but it doesn't seem that simple at all. Because there are dataIO separating everywhere in the code, it is nearly impossible to backtrace all the operations that need dataIO, not to mention that we need to let yt know when should it wait at libyt dataIO. And this seems to be save function specific, not a general approach.
If we wish to make it more general under these constraints, we need to find a way to make root rank calls an interruption at other rank, so that they know when should they be at libyt dataIO.

Notes and Things Worth Consideration:

• Will operations other than dataIO do redundant jobs on non-root rank when without wrapping is_root or only_on_root ?