The DASHMM Accelerated Adaptive Fast Multipole Poisson-Boltzmann (DAFMPB) package computes the numerical solution of the linearlized Poisson-Boltzmann equation that describes electrostatic interactions of molecular systems in ionic solutions.
The linearized Poisson-Boltzmann equation is reformulated as a boundary integral equation and is subsequently discretized using the node-patch scheme. The resulting linear system is solved using GMRES. Within each iteration, the matrix-vector multiplication is accelerated using the DASHMM library.
DAFMPB depends on two external libraries: DASHMM and HPX-5. DASHMM leverages the global address space of the HPX-5 runtime system to provide a unified evaluation of the multipole methods on both shared and distributed memory computers. This enables the latest version of AFMPB to operate on distributed memory computers while at the same time maintaining backward compatibility on shared memory computers.
Version 4.1.0 of HPX-5 is available from the contrib
directory.
DASHMM is automatically downloaded by AFMPB when the application is built.
Users must install HPX-5 on their systems before installing the DAFMPB solver. For users who use DAFMPB on shared memory computers only, HPX-5 can be built in the following steps
> cd /path/to/hpx
> ./configure --prefix=/path/to/install
> make
> make install
For users who use DAFMPB on distributed memory computers, HPX-5 currently specifies two network interfaces to choose from:
- the
ISend/IRecv
interface with the MPI transport - the Put-with-completion (PWC) interface with the Photon transport.
HPX-5 can be built with either transport.
To configure HPX-5 with MPI network, one adds --enable-mpi
to the configure
line. The configuration will search for the appropriate way to include and link
to MPI
- HPX-5 will try and see if
mpi.h
andlibmpi.so
are available with no additional flags. - HPX-5 will test for an
mpi.h
and-lmpi
in the currentC_INCLUDE_PATH
and{LD}_LIBRARY_PATH
. - HPX-5 will look for an
ompi pkg-config
package.
To configure HPX-5 with the Photon network, one adds --enable-photon
to the
configure line. HPX-5 does not provide its own distributed job launcher, so it
is necesary to also use either the --enable-mpi
or --enable-pmi
option in
order to build support for mpirun
or aprun
bootstrapping. Note that if you
are building with the Photon network, the libraries for the given network
interconnect you are targeting need to be present on the build system. The two
supported interconnects are InfiniBand (libverbs
and librdmacm
) and Cray's
GEMINI and ARIES via uGNI (libugni
). On Cray machines you need to include
PHOTON_CARGS="--enable-ugni"
to the configure line so that Photon builds with
uGNI support. Finally, the --enable-hugetlbfs
option causes HPX-5 heap to be
mapped with huge pages, which is necessary for larger heaps on some Cray Gemini
machines.
Once HPX-5 is installed, the DAFMPB package can be built in the following steps:
> mkdir dafmpb-build
> cd dafmpb-build
> cmake ../dafmpb
> make
This put the executable dafmpb
under dafmpb/example
directory.
The minimum input to dafmpb
is the PQR file.
DAFMPB can read meshes generated from MSMS or TMSMesh. If no input mesh is provided, DAFMPB will invoke the built-in surface meshing routine.
- Example 1, use built-in mesh routine
> ./dafmpb --pqr-file=GLY.pqr
- Example 2, use mesh generated by MSMS
> ./dafmpb --pqr-file=GLY.pqr --mesh-format=1 --mesh-file=GLY.pqr-mesh.data-d20-r0.5
- Example 3, use mesh generated by TMSmesh
> ./dafmpb --pqr-file=fas2.pqr --mesh-format=2 --mesh-file=fas2.off
A list of command line options to the program can be found by issuing
> ./dafmpb --help
To launch dafmpb
on a cluster with Slurm workload manager, a job script looks
like this
#! /bin/bash -l
#SBATCH -p queue
#SBATCH -N 2
#SBATCH -t 00:10:00
srun -n 2 -c 48 ./dafmpb --pqr-file=fas2.pqr.ext --mesh-format=2 --mesh-file=fas2.off --hpx-threads=24
The -c
option equals to the number of cores Slurm sees on each compute node
and the --hpx-threads
option equals the number of physical cores available on
the compute node.
If the above cluster were using the PBS workload manager, the script looks like this
#! /bin/bash -l
#PBS -l nodes=2:ppn=48
aprun -n 2 -d 48 ./dafmpb ...
Finally, the --potential-file
option allows user to specify an output stream
to hold the results of the potentials. The output can be then visualized using
VCMM.