Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

local (especially Jupyter Notebook) dev / debug? #45

Open
m-k-S opened this issue Oct 19, 2024 · 3 comments
Open

local (especially Jupyter Notebook) dev / debug? #45

m-k-S opened this issue Oct 19, 2024 · 3 comments

Comments

@m-k-S
Copy link

m-k-S commented Oct 19, 2024

Hi, I'm very interested in using Psiflow to set up MD experiments but I'm trying to get it running on my local machine before plugging it into e.g. the HPC cluster I have access to. However, I have a lot of issues doing this. For example, when I try to run the H2 static/dynamic example, I get:

RuntimeError                              Traceback (most recent call last)
Cell In[8], [line 63](vscode-notebook-cell:?execution_count=8&line=63)
     [54](vscode-notebook-cell:?execution_count=8&line=54)     return frequencies[-1] * second / (100 * _c)
     [58](vscode-notebook-cell:?execution_count=8&line=58) geometry = Geometry.from_data(
     [59](vscode-notebook-cell:?execution_count=8&line=59)     numbers=np.ones(2),
     [60](vscode-notebook-cell:?execution_count=8&line=60)     positions=np.array([[0, 0, 0], [0.8, 0, 0]]),
     [61](vscode-notebook-cell:?execution_count=8&line=61)     cell=None,
     [62](vscode-notebook-cell:?execution_count=8&line=62) )
---> [63](vscode-notebook-cell:?execution_count=8&line=63) mace = MACEHamiltonian.mace_mp0()
     [65](vscode-notebook-cell:?execution_count=8&line=65) dynamic = frequency_dynamic(geometry, mace)
     [66](vscode-notebook-cell:?execution_count=8&line=66) static = frequency_static(geometry, mace)

File ~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/hamiltonians.py:416, in MACEHamiltonian.mace_mp0(cls, size)
    [411](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/hamiltonians.py:411) urls = dict(
    [412](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/hamiltonians.py:412)     small="https://github.com/ACEsuit/mace-mp/releases/download/mace_mp_0/2023-12-10-mace-128-L0_energy_epoch-249.model",  # 2023-12-10-mace-128-L0_energy_epoch-249.model
    [413](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/hamiltonians.py:413)     large="https://github.com/ACEsuit/mace-mp/releases/download/mace_mp_0/2023-12-03-mace-128-L1_epoch-199.model",
    [414](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/hamiltonians.py:414) )
    [415](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/hamiltonians.py:415) assert size in urls
--> [416](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/hamiltonians.py:416) parsl_file = psiflow.context().new_file("mace_mp_", ".pth")
    [417](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/hamiltonians.py:417) urllib.request.urlretrieve(
    [418](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/hamiltonians.py:418)     urls[size],
    [419](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/hamiltonians.py:419)     parsl_file.filepath,
    [420](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/hamiltonians.py:420) )
    [421](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/hamiltonians.py:421) return cls(parsl_file, {})
...
    [598](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/execution.py:598)     if cls._context is None:
--> [599](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/execution.py:599)         raise RuntimeError("No ExecutionContext is currently loaded")
    [600](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/execution.py:600)     return cls._context

RuntimeError: No ExecutionContext is currently loaded

And when I try to run the basic geometry example, I get:

AssertionError                            Traceback (most recent call last)
Cell In[7], [line 7](vscode-notebook-cell:?execution_count=7&line=7)
      [3](vscode-notebook-cell:?execution_count=7&line=3) from psiflow.geometry import Geometry
      [6](vscode-notebook-cell:?execution_count=7&line=6) # a simple H2 molecule in vacuum
----> [7](vscode-notebook-cell:?execution_count=7&line=7) geometry = Geometry.from_string(
      [8](vscode-notebook-cell:?execution_count=7&line=8) '''
      [9](vscode-notebook-cell:?execution_count=7&line=9)     2
     [10](vscode-notebook-cell:?execution_count=7&line=10)     H 0.0 0.0 0.0
     [11](vscode-notebook-cell:?execution_count=7&line=11)     H 0.0 0.0 0.8
     [12](vscode-notebook-cell:?execution_count=7&line=12) ''')
     [14](vscode-notebook-cell:?execution_count=7&line=14) # the same H2 molecule using ase Atoms
     [15](vscode-notebook-cell:?execution_count=7&line=15) atoms = ase.Atoms(
     [16](vscode-notebook-cell:?execution_count=7&line=16)             numbers=[1, 1, 1],
     [17](vscode-notebook-cell:?execution_count=7&line=17)             positions=[[0, 0, 0], [0, 0, 0.8]],
     [18](vscode-notebook-cell:?execution_count=7&line=18)             pbc=False,
     [19](vscode-notebook-cell:?execution_count=7&line=19)             )

File ~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/geometry.py:184, in Geometry.from_string(cls, s, natoms)
    [180](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/geometry.py:180) else:
    [181](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/geometry.py:181)     lines = s.rstrip().split(
    [182](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/geometry.py:182)         "\n"
    [183](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/geometry.py:183)     )  # i-PI nonperiodic starts with empty -> rstrip!
--> [184](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/geometry.py:184) assert len(lines) == natoms + 1
    [185](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/geometry.py:185) comment = lines[0]
    [186](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/geometry.py:186) comment_dict = key_val_str_to_dict_regex(comment)

AssertionError:

Are Psiflow scripts not intended to be developed locally? Is there any guide on how to get it working in a Jupyter notebook? I am running Python 3.10.15, a fresh Conda virtual env and installed Psiflow via pip.

@svandenhaute
Copy link
Member

They are intended to be developed locally, but running them in a Jupyter notebook is perhaps nontrivial since Parsl requires to wrap the whole execution inside a context manager, and psiflow therefore also adopts that structure.

def main():
    # stuff
    pass

if __name__ == '__main__':
    with psiflow.load():
        main()

From your error, it looks like the psiflow.load() command was not executed in the notebook?

@m-k-S
Copy link
Author

m-k-S commented Oct 20, 2024

Sorry, yes, that was an oversight on my part. And ok, that makes sense re: Parsl. Is there any way to pass the contents of an equivalent YAML file to the main psiflow object in a Jupyter Notebook?

Also, even with the psiflow.load() context, I still get the error:

AssertionError                            Traceback (most recent call last)
Cell In[40], [line 56](vscode-notebook-cell:?execution_count=40&line=56)
     [53](vscode-notebook-cell:?execution_count=40&line=53)     frequencies = compute_frequencies(hessian, minimum).result()
     [54](vscode-notebook-cell:?execution_count=40&line=54)     return frequencies[-1] * second / (100 * _c)
---> [56](vscode-notebook-cell:?execution_count=40&line=56) with psiflow.load():
     [57](vscode-notebook-cell:?execution_count=40&line=57)     geometry = Geometry.from_data(
     [58](vscode-notebook-cell:?execution_count=40&line=58)         numbers=np.ones(2),
     [59](vscode-notebook-cell:?execution_count=40&line=59)         positions=np.array([[0, 0, 0], [0.8, 0, 0]]),
     [60](vscode-notebook-cell:?execution_count=40&line=60)         cell=None,
     [61](vscode-notebook-cell:?execution_count=40&line=61)     )
     [62](vscode-notebook-cell:?execution_count=40&line=62)     mace = MACEHamiltonian.mace_mp0()

File ~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/execution.py:586, in ExecutionContextLoader.load(cls, psiflow_config)
    [584](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/execution.py:584) assert len(sys.argv) == 2
    [585](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/execution.py:585) path_config = psiflow.resolve_and_check(Path(sys.argv[1]))
--> [586](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/execution.py:586) assert path_config.exists()
    [587](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/execution.py:587) assert path_config.suffix in [".yaml", ".yml"], (
    [588](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/execution.py:588)     "the execution configuration needs to be specified"
    [589](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/execution.py:589)     " as a YAML file, but got {}".format(path_config)
    [590](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/execution.py:590) )
    [591](https://file+.vscode-resource.vscode-cdn.net/Users/muon/Documents/materials/curie/~/miniforge3/envs/md/lib/python3.10/site-packages/psiflow/execution.py:591) with open(path_config, "r") as f:

AssertionError:

And this is with the following config file, note gpu explicitly marked as false. I believe the assertion expects gpu to always be set to True?

retries: 0
ModelEvaluation:
  gpu: false
  use_threadpool: true
  max_simulation_time: 0.4```

@svandenhaute
Copy link
Member

Yes, you can just pass the contents of the YAML file as a dictionary to the psiflow.load() call. In your case this would have to be:

import psiflow


if __name__ == '__main__':
    config_dict = {
        'retries': 0,
        'ModelEvaluation': {
            'gpu': False,
            'use_threadpool': True,
            'max_simulation_time': 0.4,
        },
        'ModelTraining': {
            'gpu': True,
            'use_threadpool': True,
        },
    }
    with psiflow.load(config_dict):
        print('test')

At the moment, users are forced to also specify ModelTraining with gpu: true -- which obviously doesn't make sense if no training is required. I'll fix this asap.

Does it raise any errors when you use the above config in your Jupyter environment? I'll talk to the people at Parsl to ask what the recommended way is for users to execute workflows in notebooks. We should put effort in making this more straightforward.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants