[mpich-discuss] srun env passing

[Dave Goodell]

Try rebuilding your MPICH omitting the "--with-pmi=slurm".  Instead, you can use hydra's built-in support for slurm.  Some information is available here: http://wiki.mpich.org/mpich/index.php/Using_the_Hydra_Process_Manager

I can't remember exactly what will be auto detected and what won't (Pavan, can you help me out here?).  You might need to set these environment variables:

HYDRA_LAUNCHER=slurm
HYDRA_RMK=slurm

Then use "mpiexec" instead of "srun".

-Dave

On Feb 4, 2013, at 9:51 AM CST, Biddiscombe, John A. wrote:

> Hello,
> I’m working on someone else’s cluster and the mpi is not well setup (i.e. not working).
>  
> I compiled my own mpich2 using
> ./configure --with-pmi=slurm --with-pm=no --prefix=/home/biddisco/apps/mpich2-1.5 --enable-shared --disable-static
> And a simple hello world executed using srun, works as expected and I get the ranks from all nodes responding correctly.
>  
> When I used mpiexec previously, I ran an MPMD command like this
>  
> mpiexec -n 4 -env DISPLAY :0.0 /home/biddisco/build/pv-38/bin/pvserver -rc -ch=128.178.187.20 -sp=11111 : -n 4 -env DISPLAY :0.1 /home/biddisco/build/pv-38/bin/pvserver -rc -ch=128.178.187.20 -sp=11111 : -n 4 -env DISPLAY :0.2 /home/biddisco/build/pv-38/bin/pvserver -rc -ch=128.178.187.20 -sp=11111
>  
> I need each of the 3 sets of mpi ranks to have a different DISPLAY environment variable to correctly utilise the GPUs.
>  
> Unfortunately, the mpiexec on the machine does not work with slurm and I’d like to use
> srun –n XXX –env XXX /program : –n XXX –env YYY /program ...
> etc etc
>  
> but srun doesn’t have an option to pass command line environment vars to the mpi processes. I just can’t find a way of launching an MPMD job using srun with different env vars.
>  
> Is there another way?
>  
> Thanks
>  
> JB
>  
>  
> --
> John Biddiscombe,                        email:biddisco @.at.@ cscs.ch
> http://www.cscs.ch/
> CSCS, Swiss National Supercomputing Centre  | Tel:  +41 (91) 610.82.07
> Via Trevano 131, 6900 Lugano, Switzerland   | Fax:  +41 (91) 610.82.82
>  
> _______________________________________________
> discuss mailing list     discuss at mpich.org
> To manage subscription options or unsubscribe:
> https://lists.mpich.org/mailman/listinfo/discuss