Investigate incompatibility of MUSIC with mpi4py version 4

[JCGoran]

In #3017 we pinned the version of mpi4py to <4 since MUSIC does not build properly under version 4 and above, review what can be done to make it compatible.

[JCGoran]

The traceback seems to be (taken from https://github.com/neuronsimulator/nrn/actions/runs/11139748696/job/30957040687):

cd ../pymusic; /home/runner/work/_temp/music-venv/bin/python -c \
"from Cython.Build import cythonize; \
cythonize('pymusic.pyx', verbose=1)"
cd ../pymusic; /home/runner/work/_temp/music-venv/bin/python -c \
"from Cython.Build import cythonize; \
cythonize('pybuffer.pyx', verbose=1)"
/home/runner/work/_temp/music-venv/lib/python3.13/site-packages/Cython/Compiler/Main.py:381: FutureWarning: Cython directive 'language_level' not set, using '3str' for now (Py3). This has changed from earlier releases! File: /home/runner/work/_temp/MUSIC/pymusic/pybuffer.pxd
  tree = Parsing.p_module(s, pxd, full_module_name)
warning: pybuffer.pxd:8:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
/home/runner/work/_temp/music-venv/lib/python3.13/site-packages/Cython/Compiler/Main.py:381: FutureWarning: Cython directive 'language_level' not set, using '3str' for now (Py3). This has changed from earlier releases! File: /home/runner/work/_temp/MUSIC/pymusic/pymusic.pxd
  tree = Parsing.p_module(s, pxd, full_module_name)
warning: pymusic.pxd:8:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
Compiling pybuffer.pyx because it changed.
[1/1] Cythonizing pybuffer.pyx
warning: pymusic.pxd:140:37: Declarations should not be declared inline.
warning: pymusic.pxd:141:37: Declarations should not be declared inline.
warning: pymusic.pxd:142:25: Declarations should not be declared inline.
warning: pymusic.pxd:143:25: Declarations should not be declared inline.
warning: pymusic.pxd:144:26: Declarations should not be declared inline.
warning: pymusic.pxd:145:51: Declarations should not be declared inline.
warning: pymusic.pxd:147:59: Declarations should not be declared inline.
warning: pymusic.pxd:149:59: Declarations should not be declared inline.
warning: pymusic.pxd:151:59: Declarations should not be declared inline.
warning: pymusic.pxd:154:59: Declarations should not be declared inline.
warning: pymusic.pxd:156:75: Declarations should not be declared inline.
warning: pymusic.pxd:158:59: Declarations should not be declared inline.
warning: pymusic.pxd:160:59: Declarations should not be declared inline.
warning: pymusic.pxd:162:71: Declarations should not be declared inline.
warning: music/pybuffer.pxd:8:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
Compiling pymusic.pyx because it changed.
[1/1] Cythonizing pymusic.pyx
make  install-recursive
make[2]: Entering directory '/home/runner/work/_temp/MUSIC/pymusic'
Making install in examples
make[3]: Entering directory '/home/runner/work/_temp/MUSIC/pymusic/examples'
make[4]: Entering directory '/home/runner/work/_temp/MUSIC/pymusic/examples'
make[4]: Nothing to be done for 'install-exec-am'.
 /usr/bin/mkdir -p '/opt/MUSIC/share/music-1.2.1/examples/python'
 /usr/bin/install -c -m 644 eventlogger.py event.music eventsource.py helloworld.music messagelogger.py message.music message.py messagesource.py receivers.py senders.py test.music testsink.py testsource.py '/opt/MUSIC/share/music-1.2.1/examples/python'
make[4]: Leaving directory '/home/runner/work/_temp/MUSIC/pymusic/examples'
make[3]: Leaving directory '/home/runner/work/_temp/MUSIC/pymusic/examples'
make[3]: Entering directory '/home/runner/work/_temp/MUSIC/pymusic'
/bin/bash ../libtool  --tag=CXX   --mode=compile mpicxx -DHAVE_CONFIG_H -I. -I..   -I/opt/hostedtoolcache/Python/3.13.0-rc.2/x64/include/python3.13 -I../src -I../src  -fno-strict-aliasing -g -O3 -g -O3 -std=c++11 -MT pymusic_la-pymusic.lo -MD -MP -MF .deps/pymusic_la-pymusic.Tpo -c -o pymusic_la-pymusic.lo `test -f 'pymusic.cpp' || echo './'`pymusic.cpp
/bin/bash ../libtool  --tag=CXX   --mode=compile mpicxx -DHAVE_CONFIG_H -I. -I..   -I/opt/hostedtoolcache/Python/3.13.0-rc.2/x64/include/python3.13 -I../src  -fno-strict-aliasing -g -O3 -g -O3 -std=c++11 -MT pybuffer_la-pybuffer.lo -MD -MP -MF .deps/pybuffer_la-pybuffer.Tpo -c -o pybuffer_la-pybuffer.lo `test -f 'pybuffer.cpp' || echo './'`pybuffer.cpp
libtool: compile:  mpicxx -DHAVE_CONFIG_H -I. -I.. -I/opt/hostedtoolcache/Python/3.13.0-rc.2/x64/include/python3.13 -I../src -I../src -fno-strict-aliasing -g -O3 -g -O3 -std=c++11 -MT pymusic_la-pymusic.lo -MD -MP -MF .deps/pymusic_la-pymusic.Tpo -c pymusic.cpp  -fPIC -DPIC -o .libs/pymusic_la-pymusic.o
libtool: compile:  mpicxx -DHAVE_CONFIG_H -I. -I.. -I/opt/hostedtoolcache/Python/3.13.0-rc.2/x64/include/python3.13 -I../src -fno-strict-aliasing -g -O3 -g -O3 -std=c++11 -MT pybuffer_la-pybuffer.lo -MD -MP -MF .deps/pybuffer_la-pybuffer.Tpo -c pybuffer.cpp  -fPIC -DPIC -o .libs/pybuffer_la-pybuffer.o
pybuffer.cpp:1709:3: error: ‘MPI_Session’ does not name a type
 1709 |   MPI_Session ob_mpi;
      |   ^~~~~~~~~~~
make[3]: *** [Makefile:564: pybuffer_la-pybuffer.lo] Error 1
make[3]: *** Waiting for unfinished jobs....
pymusic.cpp:1818:3: error: ‘MPI_Session’ does not name a type
 1818 |   MPI_Session ob_mpi;
      |   ^~~~~~~~~~~

Note that mpi4py 3.1 does not build properly OOTB on Python 3.13, however, thanks to this comment, it can be built manually with the following:

• clone mpi4py branch maint-3.1
• run ./conf/cython.sh from the top level dir
• run pip install .

Unfortunately, this cannot be converted automatically to a simple pip install since there's no way to run a pre/post command with it, but it may be possible to modify the CI to do it (though the installation instructions would then need to be modified as well).

[dalcinl]

Folks, the build issue seems to be related to the MPI_Session type.
You are using an MPI implementation/version that does not support MPI 4, thus no MPI_Session
For a relatively easy workaround, see mpi4py/mpi4py#525 (comment).