From: Matthias Maier
- MUMPS
- should be readily packaged by most Linux distributions.
- Don't forget to install a develpment version of the library.
-
- Below is a detailed instruction on how to compile and install
- MUMPS and some of its dependencies by hand (for
- the case you wish to do so).
-
- In the following, we describe the compilation process of
- MUMPS and its dependencies starting
- with BLACS and following with
- SCALAPACK.
-
- The compilation of MUMPS and its dependencies
- is not as straightforward is we would like, however, it is not
- so difficult that we can not talk about it. That said, we cannot
- discuss every possible choice of parameters, because this is too
- involved (and we would probably much rather be doing something
- productive with the interface and deal.II). Thus
- we give only a short walk-through mentioning the most important
- parameters needed for successful compliation and linking.
- The compilation scheme involves repeating the same three steps
- for each piece of software we want installed: (i) Selecting an
- appropriate makefile; (ii) Modifying symbols in this makefile to
- suit the environment; and (iii) running make to actually compile
- the software.
- Note: If you are using a system, which provides
- Intel's MKL, you should only have to compile
- MUMPS, since BLACS and
- SCALPACK are already included in the
- MKL. Therefore you can jump directly to
- section 3, where the compilation of MUMPS is
- considered. Otherwise start right below with the installation
- of BLACS.
- The BLACS library
- provides basic linear algebra routines together with a
- communication model such that they can be used in parallel. The
- compilation of this library is highly dependent on the system
- and software used and we can only give a general guide through
- the configuration process.
- The library can be found on the
- official BLACS
- webpage. We require the three
- files We start by opening Bamke.inc in our favourite text editor and
- taking a look at the following three sections:
-
- Interfacing deal.II
- to MUMPS
-
- How to compile and install MUMPS by hand
-
- 1. BLACS
-
- mpiblacs.tgz
,
- mpiblacs-patch03.tgz
, and
- blacstester.tgz
. After extraction of these
- three files in the same directory, we find a subfolder
- named BMAKES
in that directory. Inside this
- subfolder there are lots of different Bmake-files for
- different architectures. The naming of the files are
- conventional: Bmake.XXX-YYY
,
- where XXX
stands for the message passing software
- of your system
- (e.g. MPI, PVM)
- and
- YYY
for the system architecture (e.g. LINUX,
- SUN4, SUN4SOL). Pick the corresponding file for your combination
- of message passing software and system architecture, copy it to
- the root directory of BLACS, and rename it
- Bmake.inc
. Now that we have the correct makefile,
- we have to adapt it to the system we are using.
-
-
- SECTION 1: PATHS AND LIBRARIES
set the
- MPI environment correctly by changing the
- variable MPIDIR
to the directory, where your
- MPI installation is located
- in. Furthermore you will probably have to adjust the
- variables MPILIBdir
,
- MPIINCdir
and MPILIB
.
- SECTION 2: BLACS INTERNALS
- the most important variables are INTFACE
and
- TRANSCOMM
. If you are using OpenMPI, then set
- these as follows: INTFACE = -DAdd_
- and TRANSCOMM = -DUseMpi2
. Otherwise, consult
- the
- BLACS
- webpage for details on how to set these values
- correctly.
- SECTION 3:
- Compilers
. Here we set the
- variables F77
and CC
to
- correspond to the MPI Fortran and CC
- compiler respectively, e.g.
- mpif77
and mpicc
. An important
- issue here are the variables F77FLAGS
- and CCFLAGS
. It is absolutely vital to add the
- flag -fPIC
to both of them, otherwise it will
- not be possible to link deal.II with the
- BLACS library correctly when shared
- libraries are enabled.
-
Assuming we have done all of that correctly, we go back to
- the BLACS root directory and compile the
- library with the command make mpi
.
-
The SCALAPACK
- library provides high-performance linear algebra routines for
- distributed-memory message-passing MIMD
- computers. The library can be found on the
- official SCALAPACK
- webpage. We require the
- file scalapack.tgz
version <= 1.8.0 only.
-
After extracting the file and changing to the directory which
- was created, we find a file
- called SLmake.inc.example
and rename it
- to SLmake.inc
. In this file we have to set a few
- parameters as we did before with BLACS'
- own Bmake.inc
.
-
-
home
. Here you have to take care, that
- this points to the SCALAPACK directory
- (i.e. where you are right now). Next, the variable
- BLACSdir
points to the directory, where your
- BLACS libraries are located. Then set the
- variable SMPLIB
to the MPI
- library.
- BLACSFINIT
,
- BLACSCINIT
, and
- BLACSLIB
.
-
- BLACSFINIT
has to point to your
- F77init-library of BLACS. Check the
- name of the BLACS library file
- containing F77init
in its name and put in
- this name here;
- BLACSCINIT
has to point to
- the BLACS library file
- containing Cinit
in its name;
- BLACSLIB
points to the remaining
- BLACS library file.
- F77
and CC
should point to
- your MPI Fortran and CC compiler
- respectively. Also at this place we have to add the
- flag -fPIC
to the F77FLAGS
and to
- the CCFLAGS
. To the variable CDEFS
- you have to add the value of the
- variable INTFACE
in the configuration
- file Bmake.inc
in BLACS.
- BLAS
and
- LAPACK
point to your BLAS
- and LAPACK libraries compiled on your
- system.
- If all variables are set correctly we can go ahead and compile
- the SCALAPACK library with the
- command make
.
-
Now we come to the main program for which we are doing all - this. We want to install the parallel sparse direct - solver MUMPS. -
- - After downloading the solver from its
- official webpage,
- extracting it, and changing to the directory which was created, we
- find a subfolder named
- Make.inc
where there are bunch of different
- Makefiles for different machine architecture. Here it is
- important to realize that for each architecture there are two
- different files — one with the ending .SEQ
- and one without. The makefiles with the extension
- .SEQ
are the ones required for the sequential
- version of MUMPS. Since we want to use MUMPS
- in parallel, we take the file
- without this extension which looks suitable for the
- architecture we are building on and copy this into
- the MUMPS root directory and rename it
- as Makefile.inc
.
-
As before, we open the makefile Makefile.inc
in
- our favourite text editor and start modifying variables:
-
-
CC
should point to
- your MPI CC
- compiler, e.g. mpicc
. The
- variables FC
and FL
should both
- point to your MPI Fortran90 compiler,
- e.g. mpif90
.
- SCALAP
. This one should point to your
- SCALAPACK0 and BLACS
- installations as follows:
- -L/XXX -lscalapack -L/YYY -lblacs -lblacsCinit
- -lblacsF77init -lblacs -lblacsCinit
;
- where XXX
stands for the directory containing
- the SCALAPACK library
- and YYY
for the directory containing the
- BLACS library. The
- variable INCPAR
should point to
- the MPI include directory.
- LIBPAR
which
- should contain the same value as the
- variable SCALAP
plus a pointer to
- the MPI library. Therefore we propose to
- set it to the following value: $(SCALAP) -L/XXX
- -lmpi
, where XXX
stands for the
- directory containing your MPI
- libraries. Please check, if the
- variable LIBBLAS
points to your BLAS
- installation. The variable
- CDEFS
has to contain the same value as the variable
- INTFACE
of
- BLACS' own Bmake.inc
. If you
- are using OpenMPI this should be the
- value -DAdd_
.
- -fPIC
to the
- three compiler flag variables OPTF
,
- OPTC
, and OPTL
which follow
- immediately after the above. Without adding this last flag it
- will not be possible to link deal.II with
- MUMPS as a shared library.
-
- Now we are done with configuration and can
- compile MUMPS with the
- command make
.
-
- Support for MUMPS will be enabled automatically - if a system wide installation of MUMPS - can be found. To use a self compiled version, specify -
- - -DMUMPS_DIR=/path/to/mumps -on the command line when invoking
cmake
.
-
-
- - For SCALAPACK and BLACS a hint - - if necessary - may be given by -
- - -DSCALAPACK_DIR=/path/to/scalapack - -DBLACS_DIR=/path/to/blacs -- - -
- You can override the autodetection by manually specifying -
- - -DDEAL_II_WITH_MUMPS=OFF|ON -. - - -
- Note: Throughout this description of the compilation process
- of MUMPS we have emphasised adding the compiler
- flag -fPIC
. This is a definite requirement if we
- are compiling deal.II with shared libraries
- (which is the default). If we had preferred to be compiling
- deal.II without shared libraries, that's ok too;
- in that case we would do exactly the same thing as described above,
- but this time omitting the -fPIC
flag from the scheme.
-
SparseDirectMUMPS
has been removed. The MUMPS solver is
+ still available through the Trilinos or PETSc interfaces. Alternatively,
+ there is SparseDirectUMFPACK
, which has a similar interface.
+
- MUMPS
- (MUltifrontal Massively Parallel sparse direct Solver)
- is a sparse direct solver that can factorize matrices within or
- across nodes of a cluster.
- MUMPS
- should be readily packaged by most Linux distributions.
- (Don't forget to install a development version of the library).
- To use a self compiled version, provide the
- -DMUMPS_DIR=/path/to/mumps
argument to cmake
.
- For a detailed description on how to compile MUMPS (and
- some dependencies) and linking with deal.II, see
- this page.
-