mpichhosts has been rewritten to accommondate new changes in MPICH2 v1.2.1. system("smpd -get hosts", intern=TRUE) is used to find MPICH2's hosts information instead of registry query. Due to a bug in R's function system, it returns empty under both Windows 7 32bit and 64bit. This effects the outcome of mpi.universe.size. If you run Rmpi with multiple windows PCs, please specify nslaves in mpi.spawn.Rslaves under Windows 7 master. Hopefully this bug will be fixed in next release R 2.10.2. When hosts information is not specified in MPICH2 (default is used), mpichhosts() will find total number of CPUs from registry query.
Add mpi.parMM and mpi.iparMM functions for parallel matrix products. mpi.parMM is a clone of parMM from snow package. It has a load balancing parmeter job.num. Although parallel matrix product can be achieved by mpi.parApply(A, 1, "%*%", B), mpi.parMM is more efficient and faster. Choose job.num to be 2 to 4 times of total number of slaves to futher speed up of huge matrix product. mpi.iparMM is a nonblocking version of mpi.parMM without load balancing feature.
Fix a bug in mpi.iapplyLB.
configure.ac has been completely rewriteen by Prof. Brian Ripley. Now it allows --with-Rmpi-include= and --with-Rmpi-libpath= options so that MPI's include and lib paths don't need to have the same root. The original option --with-Rmpi= is still available. Another feature includes the option --with-Rmpi-type= to specify OPENMPI, LAM, or MPICH. Thanks "Prof. Brian Ripley" <ripley at stats.ox.ac.uk>.
Fix a bug in mpi.comm.dup. Thanks "George Ostrouchov" <ostrouchovg at ornl.gov>.
Fix "Unknown package(s) ‘serialize’ in Rd xrefs" warning, compiled under 2.10.0 or higher.
Add nonblocking mpi.iapply, mpi.iapplyLB, mpi.iparApply, mpi.iparLapply, mpi.iparSapply, mpi.iparCapply, mpi.iparRapply, and mpi.iparReplicate functions. In those functions, mpi.iprobe is used while master is waiting for slave results. After mpi.iprobe returns FALSE, master went sleep with duration specified by sleep argument (default 0.001 sec or 1ms). Now, under OpenMPI, master will use little CPU cycles while waiting for slave results.
Fix a segment fault under OpenMPI 1.3 and Debian 64bit. Thanks "Jeff Squyres" <jsquyres at cisco.com> and "Dirk Eddelbuettel" <edd at debian.org> for the fixing.
Add C level function mpidist. .Call("mpidist",PACKAGE="Rmpi") returns 1 if it is OpenMPI; 2 if LAM and 3 if MPICH.
Modify .First.lib so that dynamic library will be loaded first. Then in case LAM is detected, lamboot might be used.
Add additinoal detections of OpenMPI at various locations, including /usr/lib/openmpi, suggested by "Dirk Eddelbuettel" <edd at debian.org>.
Modify mpi.parApply so that parallel matrix product, like mpi.parApply(A, 1, "%*%", B), can be used. This was suggested by "Martin Morgan" < mtmorgan at fhcrc org>
Modify mpi.apply so that mpi.scatter.Robj is used instead of mpi.send.Robj. This may result some improvements with large datasets.
Fix a bug in mpi.scatter.Robj and add a new function mpi.scatter.Robj2slave which is more efficient than using mpi.parLapply to distribute components of a list to slaves.
sleep.exe is removed from inst directory to avoid the possibility of virus infection. So is the corresponding line in Rslaves.bat.
Modify Rprofile in inst directory so that it will not generate errors under OpenMPI when leaving slaves.
dlopen has been used to load libmpi.so explicitly. This is
mainly useful for Rmpi under OpenMPI where one might see many error messengers:
mca: base: component_find: unable to open osc pt2pt: file
not found (ignored)
if libmpi.so is not loaded with RTLD_GLOBAL flag.
In Rslave.sh, [ $3 == "needlog" ];
is changed to [ "$3" = "needlog" ]; This might stop
those dreadful error messengers:
[: 20: ==: unexpected operator
during spawning R slaves, suggested by "Dirk Eddelbuettel" <edd at
debian.org>. Similarly, in configure.ac,
test "$MPI_ROOT"==""; is
changed to [ [ "$MPI_ROOT" = "" ] ]; Otherwise it
will not take --with-mpi argument, reported by "Shintaro KATAYAMA"
<katayama at gsc riken jp>.
All objects in C codes are properly initialized, done by "Dirk Eddelbuettel" <edd at debian.org>. This will stop warning messengers during compiling on some Linux distributions such as Fedora.
MPI_Init(&fake_argc, (char ***)(void*)&fake_argv) has been used when compiling Rmpi under MPICH otherwise loading Rmpi in R will result segmentation errors. This was suggested by "Aaron Sethman" <sethmana at mail.nih.gov>. For other systems even with MPICH2, MPI_Init((void *)0,(void *)0); is still used.
configure.ac has been reorganized. Now it will detect OpenMPI first, then LAM and finally MPICH or MPICH2. It can also detect OpenMPI or LAM on both Fedora 32bit or 64bit systems. Users can still overwrite those detections with --with-mpi argument or MPI_ROOT environment variable.
Modify mpi.spawn.Rslaves so that it will not get "System could not find the path" error during spawning under Windows.
The original .splitIndices function (copied from snow) did not split indices evenly. As suggested by my colleague Dr. McLeod <aim at stats.uwo.ca>, I used the algorithm in equal.count from lattice package to split indices evenly.
Transferring in mpi.bcast.cmd has been changed from character to raw. This might add some overhead but it will avoid any issues associated with character transfer.
All character vector transfers have been rewritten since R 2.6.0 does not allow any modifications under C level. Due to complication of creating multiple temp buffers for storing receiving buffers, only the first element in a
character vector (R char vector) is transferred. For sending buffers, #define CHAR2(x) ((char *)
CHAR(x)) has been used, suggested by Prof. Ripley <ripley at
stats.ox.ac.uk>. In addition character receive in mpi.irecv is disabled.
Add lib64/lam in search path in configure.ac. This might make Rmpi compile under FC 64bit.
Change libmpi.a to libmpi.so in configure.ac and add detection of openmpi dir so that Rmpi can be compiled under OpenMPI with full
functionality. R can be started like R -> library(Rmpi) -> mpi.spawn.Rslaves() without remote nodes. Or R can be started like "mpirun --host hostfile -np 1 R --no-save" and library(Rmpi) ->
mpi.spawn.Rslaves() bur R runs in noninteractive mode without command recall. One major problem associated with OpenMPI is that all R slaves run at 100%CPU while they are waiting for master instructions. So the implementation of MPI_BCAST in OpenMPI is different from that in LAM. Hence Rmpi under OpenMPI is not good for coding and testing. Rather it should be used for real
computations.
Add AC_PROG_CC in configure.ac to work with SUSE linux, suggested by George
Ostrouchov <ostrouchovg at ornl.gov>.
Modify configure.ac so that LAM on Mac OS X can be detected, provided that OS X version of LAM at www.lam-mpi.org is installed.
Change configure.in to configure.ac and add MPI_INCLUDE, LDFLAGS, CFLAGS, in case MPI_ROOT is supplied by users, suggested by "Dennis van
Dok" <dennisvd at nikhef.nl>.
R raw data type has been added as type 4. This is used for transferring
R objects, i.e., all R objects are serialized to raw type for transferring. This is in line with the change of serialize in R 2.4.0. Wrappers .mpi.serialize and .mpi.unserialize are used in case Rmpi is running under R 2.2.0 to R.2.3.1.
Add a few examples to several mpi functions.
Add mpi.scatter.Robj, mpi.gather.Robj, and mpi.allgather.Robj.
Change "MPI_Comm_free(&master);" in mpi_is_master of Rmpi.c to
"if (!check) MPI_Comm_free(&master);" Thanks to Martin Siegert <siegert
at sfu.ca> for fixing.
Some patch to configure.in, suggested by Aaron A. King <aaron.king at
umich.edu>, so that if MPI_ROOT is supplied as an environmental variable or is given by users, mpi.h will be searched within subfolders of MPI_ROOT.
Rename some internal functions used by slaves. They follow by the convention
.mpi.slave.xxx and hidden.
Automatically loading SPRNG in mpi.spawn.Rslaves is dropped. Instead one can use mpi.setup.sprng or mpi.setup.rngstream to setup parallel RNG on slaves.
Some codes tidy on mpi.remote.exec and mpi.parSim. As a result, mpi.wrap.fun is dropped.
Add mpichhosts to find host names on windows platform.
Modify mpi.universe.size to return length(mpichhosts())-1 under windows platform.
Add mpi.apply: an array is scattered to slaves and then a fun is applied.Add mpi.applyLB: a load balancing version of mpi.apply.
Add mpi.parApply, mpi.parLapply, mpi.parRapply, mpi.parCapply, and
mpi.parSapply: they are clones of those functions in package snow if job.num=slave.num. They are load balanced if job.num > slave.num.
Add mpi.parReplicate: wrapper for mpi.parSapply, mainly used in parallel simulation. It will eventually replace mpi.parSim due to its efficiency.
Add network map drive function in inst/Rslaves.bat. May see some error messages
if multiple R slaves are spawned on the same remote node.
Add nonblocking calls: mpi.isend, mpi.irecv, mpi.isend.Rojb, mpi.iprobe,
mpi.test, mpi.testany, mpi.testsome, mpi.testall, mpi.wait, mpi.waitany,
mpi.waitsome, mpi.waitall, mpi.cancel, and mpi.test.cancelled.
Add mpi.comm.maxsize, mpi.request.maxsize, mpi.status.maxsize,
mpi.realloc.comm, mpi.realloc.request, and mpi.realloc.status.
Rmpi can be compiled and run under MPICH2 for Windows. src/Makevars.win is
added so that Rmpi can be compiled without manual configuration if MPICH2 for Windows is installed at default place. Instructions to run Rmpi on windows platform are given in
here.
inst/Rslaves.bat windows batch file is created for spawning R slaves under windows platform (localhost only, a restriction imposed by MPICH2 for Windows).
Full sets of creating a Cartesian topology APIs are implemented.
inst/Rslaves.sh is modified so that the same R program Master R is ruining will be used by slaves Rs. This allows co-existences of different versions of R.
inst/Rprofile used by mpiexec under unix and windows platforms is modified. The main change is that sys.parent() in eval is removed. This is due to a unknown reason that the infinite loop cannot be broken if the env R_RPOFILE is used to launch multiple Rs. Objects created on R slaves are no longer global unless <<- are used, though they are accessible during the duration of R slaves. If Rprofile is copied as .Rprofile at the working directory,
sys.parent() can still be used (need to modify .Rprofile).
To run multiple Rs without using .Rprofile, add the following line R_PROFILE=${R_HOME}/library/Rmpi/Rprofile; export R_PROFILE
in the R shell script (after "export R_HOME") and rename it, say, Rmpi. Then one can use
mpiexec -n number_of_nodes Rmpi --no-save -q to launch multiple Rs (assume Rmpi is in the PATH).
Parts of nonblocking point-to-point MPI APIs are implemented at C level only. They will be released in the next version of Rmpi.
mpi.remote.fun is deprecated and mpi.remote.exec (mpi.remote.slave)
has been simplified by using do.call instead.
replicate is used in mpi.parallel.sim (mpi.remote.slave) instead of
sapply. To be consistent, mpi.parallel.sim uses cbind to combine all slave results together.
Delete .Random.seed in .First.lib if it is a user-defined RNG. Add needsprng = TRUE argument to mpi.spawn.Rslaves so that SPRNG can be loaded if there is the package rsprng. Implement mpi.parallel.sim for parallel Monte Carlo simulation. Some ideas from student projects of Mr. K. Yu, Ms. Tsai, and Mr. W. Wolterman have been used. Thanks to them for their wonderful works.
Update to require R-1.8.1 and remove the requirement for the serialise package (build into R-1.8.1).
Some minor bugs fix.
Add #ifdef MPI2 to src/Rmpi.c and change MPI_Comm_set_errhandler back to MPI_Errhandler_set for backward compatibility with MPI-1.2, suggested by Ross Boylan.
Add -DMPI2 in configure.in in case lam lib is detected. Otherwise MPI-2 related functions will be disabled.
Add detect MPI in /usr/lib/mpi or /usr/lib/mpich, hopefully to catch MPICH MPI.
Add some lines to configure.in so that configure can detect lam on Debian Linux. Thanks to Kurt Hornik <Kurt.Hornik at wu-wien.ac.at> for Debian instructions.
Internal use.
Add a function string for creating a (binary) character buffer. Remove all C++ style comments. Remove all deprecated functions. Remove mpi.barrier from slavedaemon.R (it causes slaves hanging if master crashed).
Take SPRNG out. Use rsprng package instead. Use mpi.init.sprng to start
SPRNG.
Require serialize package
mpi.send.Robj, mpi.recv.Robj, mpi.bcast.Robj, and mpi.bcast.Robj2slave use serialize to transfer R objects.
mpi.bcast.send and mpi.bcast.recv are deprecated.
mpi.bcast.send.cmd and mpi.bcast.recv.cmd are replaced by mpi.bcast.cmd
mpi.bcast.send.Robj and mpi.bcast.recv.Robj are replaced by mpi.bcast.Robj
New instructions for installing Rmpi for other MPI implementations.
Thanks to Kurt Hornik <Kurt.Hornik at wu-wien.ac.at> for Debian instructions.
Some bugs fix. Add Rprofile in inst Add a function mpi.comm.dup
Some bugs fix. Add a function mpi.quit
Initial public release