MESTI.jl uses the parallel version of MUMPS for the augmented partial factorization (APF) method, and optionally for the factorize-and-solve method. Here are the steps to install MUMPS.
Go to the MUMPS website and fill out the download request form. The MUMPS maintainers will email you the download link.
To compile the parallel version of MUMPS, you need compilation tools like make, ar, and ranlib, C and Fortran compilers, BLAS library, LAPACK library, ScaLAPACK library, and MPI library. Instructions specific to the operating system are provided below:
If you are interested in 3D systems or memory usage in 2D systems is important for you, we highly recommend you install the METIS (version 5.1.0) program for graph partitioning (not to be confused with MESTI). We can install METIS in the following steps: (a) Downloading METIS (version 5.1.0)
wget http://glaros.dtc.umn.edu/gkhome/fetch/sw/metis/metis-5.1.0.tar.gz(b) Decompress metis-5.1.0.tar.gz
tar zxvf metis-5.1.0.tar.gz(c) Setting METIS to double precision
sed -i "43s/32/64/" metis-5.1.0/include/metis.h(d) Installing METIS
cd metis-5.1.0; make config; sudo make install;Then, by default, the library file, header file, and binaries will be installed in /usr/local/lib, /usr/local/include, and /usr/local/bin. In some rare cases, your machine cannot find METIS libraries by itself when you run Julia interface for MUMPS. You can append the METIS libraries to your LD_LIBRARYP_PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$LMETISDIRLMETISDIR is the path to the folder where the METIS library is.
After installing METIS, if you set opts.use_METIS = true in mesti() or mesti2s(), MUMPS will use METIS for matrix ordering. From our experience, in 2D, AMD is usually faster when using the APF method, but METIS can sometimes reduce memory usage. In 3D, METIS is strongly recommended, which is much faster than AMD. By default, 2D systems use AMD, while 3D systems use METIS (if it is available).
Suppose you downloaded the 5.7.1 version of MUMPS to your ~/Downloads/ folder. Then, go to the folder where you want to compile MUMPS, and enter
tar zxvf ~/Downloads/MUMPS_5.7.1.tar.gz
cd MUMPS_5.7.1in terminal.
Read the file INSTALL, copy the closest Makefile.inc file in the Make.inc folder, and modify it to fit your environment and machine. Most importantly, in Makefile.inc you need to specify:
CC: the C compiler to useFC: the Fortran compiler to useFL: the Fortran linker to useLAPACK: how the Fortran compiler can link to the LAPACK librarySCALAP: how the Fortran compiler can link to the ScaLAPACK libraryLIBBLAS: how the Fortran compiler can link to the BLAS libraryRPATH_OPT: the path to shared libraries that will be built up, such as.../MUMPS_5.7.1/lib/where...is the path to MUMPS_5.7.1 folder.
Note that from our experience, RPATH_OPT must be specified to successfully install the parallel version of MUMPS on Linux and Windows.
If you installed METIS, you also need to specify
LMETISDIR: path to the folder where the METIS library isIMETIS: path to the folder where the METIS headers are
and add -Dmetis to ORDERINGSF.
Examples of Makefile.inc are provided below:
To download, click the link above, click on the "Raw" button, and right-click to save the file. If the browser adds a .txt file extension, rename it to remove the txt extension.
After done with Makefile.inc, enter
make allsharedin the terminal, which will compile the parallel version of MUMPS with single and double precision for real and complex variables (i.e., smumps, dmumps, cmumps, zmumps).
If there is no error, check if the following files have been generated in the lib folder: libsmumps.so, libdmumps.so, libcmumps.so, and libzmumps.so.
Warning messages from the Fortran compiler are normal and can be ignored.
If there is an error, read the message and try to figure out where it comes from and/or look it up and address it. Before recompiling with make allshared, be sure to type make clean first to remove files generated by the previous compile attempt.
After compiling the parallel version of MUMPS, in startup.jl we should set the Julia environment variable MUMPS_PREFIX, which is the path to your own MUMPS libraries through the terminal:
mkdir ~/.julia/config
echo 'ENV["MUMPS_PREFIX"] = ".../MUMPS_5.7.1/lib"' >> ~/.julia/config/startup.jlwhere ... is the path to MUMPS_5.7.1 folder.
When we run Julia interface for MUMPS, the machine may not find the libraries by itself. To solve those issues, please follow the steps depending on your OS:
- Linux or Windows
We can append those library paths to LD_PRELOAD before running Julia. For example, with the Intel oneAPI installed under /opt, we can type,
source /opt/intel/oneapi/mkl/latest/env/vars.sh
source /opt/intel/oneapi/mpi/latest/env/vars.sh
source /opt/intel/oneapi/compiler/latest/env/vars.sh
export LD_PRELOAD=$LD_PRELOAD:$MKLROOT/lib/intel64/libmkl_intel_lp64.so
export LD_PRELOAD=$LD_PRELOAD:$MKLROOT/lib/intel64/libmkl_intel_thread.so
export LD_PRELOAD=$LD_PRELOAD:/opt/intel/oneapi/inspector/latest/lib64/libiomp5.so
export LD_PRELOAD=$LD_PRELOAD:$MKLROOT/lib/intel64/libmkl_core.so
export LD_PRELOAD=$LD_PRELOAD:$MKLROOT/lib/intel64/libmkl_blacs_intelmpi_lp64.so
export LD_PRELOAD=$LD_PRELOAD:$MKLROOT/lib/intel64/libmkl_scalapack_lp64.soor
To use MUMPS.jl, we need to install and configure MPI.jl. Begin by installing MPI.jl and MPIPreferences.jl with the following command in Julia:
using Pkg; Pkg.add(["MPI", "MPIPreferences"])
Once the installation is complete, run the following command in Julia to configure MPI.jl:
using MPIPreferences; MPIPreferences.use_system_binary()
This will automatically find the OpenMPI library installed by Homebrew. If MPIPreferences cannot locate the library, you need to specify the library path manually:
MPIPreferences.use_system_binary(; library_names=["/path/to/open-mpi/5.x.x/lib/libmpi"])
Now, you are ready to install MESTI.jl. Please go back to install MESTI.jl.