[mpich-discuss] srun env passing

[Biddiscombe, John A.]

Hello,
I'm working on someone else's cluster and the mpi is not well setup (i.e. not working).

I compiled my own mpich2 using
./configure --with-pmi=slurm --with-pm=no --prefix=/home/biddisco/apps/mpich2-1.5 --enable-shared --disable-static
And a simple hello world executed using srun, works as expected and I get the ranks from all nodes responding correctly.

When I used mpiexec previously, I ran an MPMD command like this

mpiexec -n 4 -env DISPLAY :0.0 /home/biddisco/build/pv-38/bin/pvserver -rc -ch=128.178.187.20 -sp=11111 : -n 4 -env DISPLAY :0.1 /home/biddisco/build/pv-38/bin/pvserver -rc -ch=128.178.187.20 -sp=11111 : -n 4 -env DISPLAY :0.2 /home/biddisco/build/pv-38/bin/pvserver -rc -ch=128.178.187.20 -sp=11111

I need each of the 3 sets of mpi ranks to have a different DISPLAY environment variable to correctly utilise the GPUs.

Unfortunately, the mpiexec on the machine does not work with slurm and I'd like to use
srun -n XXX -env XXX /program : -n XXX -env YYY /program ...
etc etc

but srun doesn't have an option to pass command line environment vars to the mpi processes. I just can't find a way of launching an MPMD job using srun with different env vars.

Is there another way?

Thanks

JB


--
John Biddiscombe,                        email:biddisco @.at.@ cscs.ch
http://www.cscs.ch/
CSCS, Swiss National Supercomputing Centre  | Tel:  +41 (91) 610.82.07
Via Trevano 131, 6900 Lugano, Switzerland   | Fax:  +41 (91) 610.82.82

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20130204/085fd384/attachment.html>