This repository has been archived by the owner on Jul 22, 2024. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 54
mpi4py support #267
Comments
Forcing python to compile using xl instead of gcc also fails (py3) $ CC=xlc CXX=xlC pip install mpi4py
|
@zyndagj there is official Anaconda package for mpi4py with support for ppc64. But it relies on openmpi or mpich which are not linked with LSF, so would be limited to one node with the scheduler. https://anaconda.org/anaconda/mpi4py |
That's for the suggestion. However, multi-node execution is a requirement. It also appears that installing it with anything besides spectrum will cause issues with the distributed ML packages. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Hello,
I support an IBM AC922 system with InfiniBand, Spectrum MPI, and the xl compiler. We rely on PowerAI for delivering customizable python environments backed by an ecosystem of pre-tuned machine learning packages. We were recently asked to support mpi4py and I have been unable to compile it in a conda environment. It will compile with the system python, so xl and spectrum seem to be configured fine - issues only arise in a conda environment. I'd appreciate any help.
Thanks,
Greg
$ conda config --show
$ conda create -n pai17 python=3 powerai=1.7
$ conda activate pai17
(pai17) $ pip install mpi4py
Compiling it using the system spectrum MPI in a new python3 environment with only setuptools and wheel also fails.
$ conda create -n py3 python=3 setuptools wheel
$ conda activate py3
(py3) $ pip install mpi4py
The text was updated successfully, but these errors were encountered: