[mpich-discuss] MPICH 3.1.4 with Slurm 14.11.9
Guo, Yanfei
yguo at anl.gov
Thu Oct 15 15:51:56 CDT 2015
Hi Davide,
I have tried your configuration on our machines and I could not reproduce the problem. Can you provide some more information about how you run into this problem?
Yanfei Guo
Postdoctoral Appointee
MCS Division, ANL
On 10/15/15, 10:40 AM, "Balaji, Pavan" <balaji at anl.gov> wrote:
>Yanfei, Ken,
>
>Notice that the user is having trouble because the SLURM path is not passed to the mpich test suite (not mpich itself). My guess is that we did something weird in passing the paths downstream since setting xFLAGS didn't help the user either.
>
> -- Pavan
>
>
>
>
>On 10/15/15, 10:37 AM, "Guo, Yanfei" <yguo at anl.gov> wrote:
>
>>Then, can you check if “/usr/scheduler/slurm/lib” is in ld’s search path. You can get that by running ldconfig -v 2>/dev/null | grep -v ^$'\t’
>>
>>If not, you will need to set
>>LIBRARY_PATH=“/usr/scheduler/slurm/lib”
>>LD_LIBRARY_PATH=“/usr/scheduler/slurm/lib”
>>
>>
>>Yanfei Guo
>>Postdoctoral Appointee
>>MCS Division, ANL
>>
>>
>>
>>
>>
>>
>>
>>On 10/15/15, 10:34 AM, "Kenneth Raffenetti" <raffenet at mcs.anl.gov> wrote:
>>
>>>I think this is the right solution, but should actually be
>>>"—with-slurm=/usr/scheduler/slurm". See the help text from configure:
>>>
>>> -with-slurm=PATH specify path where slurm include directory and
>>>lib directory can be found
>>>
>>>Ken
>>>
>>>On 10/15/2015 10:25 AM, Guo, Yanfei wrote:
>>>> Can you try using “—with-slurm=/usr/scheduler/slurm/lib” when you run ./configure?
>>>>
>>>> Yanfei Guo
>>>> Postdoctoral Appointee
>>>> MCS Division, ANL
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On 10/14/15, 3:21 PM, "Davide Vanzo" <vanzod at accre.vanderbilt.edu> wrote:
>>>>
>>>>> Pavan,
>>>>> maybe you missed in the crowd of my previous email. Anyway, yes, the library is there:
>>>>>
>>>>>
>>>>> $ ls /usr/scheduler/slurm/lib
>>>>> libpmi2.a libpmi.la libslurmdb.la libslurm.so.28
>>>>> libpmi2.la libpmi.so libslurmdb.so libslurm.so.28.0.0
>>>>> libpmi2.so libpmi.so.0 libslurmdb.so.28 slurm
>>>>> libpmi2.so.0 libpmi.so.0.0.0 libslurmdb.so.28.0.0
>>>>> libpmi2.so.0.0.0 libslurm.a libslurm.la
>>>>> libpmi.a libslurmdb.a libslurm.so
>>>>>
>>>>>
>>>>> Davide
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wed, 2015-10-14 at 20:17 +0000, Balaji, Pavan wrote:
>>>>>
>>>>>
>>>>>
>>>>> Can you make sure you have SLURM's libpmi in the location you specified below (/usr/scheduler/slurm/lib)?
>>>>>
>>>>>
>>>>> -- Pavan
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> From: Davide Vanzo
>>>>> Reply-To: "discuss at mpich.org"
>>>>> Date: Wednesday, October 14, 2015 at 2:29 PM
>>>>> To: "discuss at mpich.org"
>>>>> Subject: [mpich-discuss] MPICH 3.1.4 with Slurm 14.11.9
>>>>>
>>>>>
>>>>>
>>>>> Hello all,
>>>>> I'm trying to compile MPICH 3.1.4 with GNU compilers 4.9.0 on a system with Slurm 14.11.9. I want to test the performance fall when using srun vs. mpiexec.hydra.
>>>>> I followed the guide and I added the slurm flags to configure as follows:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ./configure --with-device=ch3:nemesis --prefix=$PREFIX --enable-fortran=yes --enable-cxx --enable-shared --with-pmi=slurm --with-pm=no --with-slurm=/usr/scheduler/slurm
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> The only compiler options I'm using are:
>>>>>
>>>>>
>>>>> MPICHLIB_xFLAGS="-O3 -march=corei7"
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> MPICH builds correctly, but when I run make testing, I get the following error:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ( cd test && make testing )
>>>>> make[1]: Entering directory `/gpfs22/local/centos6/mpich/3.1.4/nehalem/gcc49/ether/source/test'
>>>>> (NOXMLCLOSE=YES && export NOXMLCLOSE && cd mpi && make testing)
>>>>> make[2]: Entering directory `/gpfs22/local/centos6/mpich/3.1.4/nehalem/gcc49/ether/source/test/mpi'
>>>>> ./runtests -srcdir=. -tests=testlist \
>>>>> -mpiexec=/usr/local/mpich/3.1.4/nehalem/gcc49/ether/bin/mpiexec -xmlfile=summary.xml \
>>>>> -tapfile=summary.tap -junitfile=summary.junit.xml
>>>>> Looking in ./testlist
>>>>> Processing directory attr
>>>>> Looking in ./attr/testlist
>>>>> Failed to build attrt; make[3]: Entering directory `/gpfs22/local/centos6/mpich/3.1.4/nehalem/gcc49/ether/source/test/mpi/attr'
>>>>> CC attrt.o
>>>>> CCLD attrt
>>>>> /usr/bin/ld: cannot find -lpmi
>>>>> collect2: ld returned 1 exit status
>>>>> make[3]: *** [attrt] Error 1
>>>>> make[3]: Leaving directory `/gpfs22/local/centos6/mpich/3.1.4/nehalem/gcc49/ether/source/test/mpi/attr'
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> although /usr/scheduler/slurm/lib/libpmi.la exists.
>>>>> So I tried to explicitly use the lib and include path in the configure options: no luck.
>>>>> I also set "-L/usr/scheduler/slurm/lib" in the xFLAGS, but no effect.
>>>>> Any suggestion?
>>>>> Please let me know if I missed some information and thank you in advance for your help.
>>>>>
>>>>>
>>>>> Best regards,
>>>>> Davide
>>>>>
>>>>>
>>>>> --
>>>>> Davide Vanzo, PhD
>>>>> Application Developer
>>>>> Advanced Computing Center for Research and Education (ACCRE)
>>>>> Vanderbilt University - Hill Center 201
>>>>> www.accre.vanderbilt.edu <http://www.accre.vanderbilt.edu/>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> discuss mailing list discuss at mpich.org
>>>>> To manage subscription options or unsubscribe:
>>>>> https://lists.mpich.org/mailman/listinfo/discuss
>>>>>
>>>> _______________________________________________
>>>> discuss mailing list discuss at mpich.org
>>>> To manage subscription options or unsubscribe:
>>>> https://lists.mpich.org/mailman/listinfo/discuss
>>>>
>>>_______________________________________________
>>>discuss mailing list discuss at mpich.org
>>>To manage subscription options or unsubscribe:
>>>https://lists.mpich.org/mailman/listinfo/discuss
>>_______________________________________________
>>discuss mailing list discuss at mpich.org
>>To manage subscription options or unsubscribe:
>>https://lists.mpich.org/mailman/listinfo/discuss
>_______________________________________________
>discuss mailing list discuss at mpich.org
>To manage subscription options or unsubscribe:
>https://lists.mpich.org/mailman/listinfo/discuss
_______________________________________________
discuss mailing list discuss at mpich.org
To manage subscription options or unsubscribe:
https://lists.mpich.org/mailman/listinfo/discuss
More information about the discuss
mailing list