<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif;">
<div>
<div>
<div><br>
</div>
<div>Can you make sure you have SLURM's libpmi in the location you specified below (/usr/scheduler/slurm/lib)?</div>
<div><br>
</div>
<div> -- Pavan</div>
<div>
<div id="MAC_OUTLOOK_SIGNATURE"></div>
</div>
</div>
</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:12pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span>Davide Vanzo<br>
<span style="font-weight:bold">Reply-To: </span>"<a href="mailto:discuss@mpich.org">discuss@mpich.org</a>"<br>
<span style="font-weight:bold">Date: </span>Wednesday, October 14, 2015 at 2:29 PM<br>
<span style="font-weight:bold">To: </span>"<a href="mailto:discuss@mpich.org">discuss@mpich.org</a>"<br>
<span style="font-weight:bold">Subject: </span>[mpich-discuss] MPICH 3.1.4 with Slurm 14.11.9<br>
</div>
<div><br>
</div>
<div>
<div>
<div>
<div style="color: rgb(0, 0, 0);">Hello all,</div>
<div style="color: rgb(0, 0, 0);">I'm trying to compile MPICH 3.1.4 with GNU compilers 4.9.0 on a system with Slurm 14.11.9. I want to test the performance fall when using srun vs. mpiexec.hydra.</div>
<div style="color: rgb(0, 0, 0);">I followed the guide and I added the slurm flags to configure as follows:</div>
<div style="color: rgb(0, 0, 0);"><br>
</div>
<div style="color: rgb(0, 0, 0);"><br>
</div>
<div style="color: rgb(0, 0, 0);">./configure --with-device=ch3:nemesis --prefix=$PREFIX --enable-fortran=yes --enable-cxx --enable-shared --with-pmi=slurm --with-pm=no --with-slurm=/usr/scheduler/slurm</div>
<div style="color: rgb(0, 0, 0);"><br>
</div>
<div style="color: rgb(0, 0, 0);"><br>
</div>
<div style="color: rgb(0, 0, 0);">The only compiler options I'm using are:</div>
<div style="color: rgb(0, 0, 0);"><br>
</div>
<div style="color: rgb(0, 0, 0);">MPICHLIB_xFLAGS="-O3 -march=corei7"</div>
<div style="color: rgb(0, 0, 0);"><br>
</div>
<div style="color: rgb(0, 0, 0);"><br>
</div>
<div style="color: rgb(0, 0, 0);">MPICH builds correctly, but when I run make testing, I get the following error:</div>
<div style="color: rgb(0, 0, 0);"><br>
</div>
<div style="color: rgb(0, 0, 0);"><br>
</div>
<div style="color: rgb(0, 0, 0);">( cd test && make testing )</div>
<div style="color: rgb(0, 0, 0);">make[1]: Entering directory `/gpfs22/local/centos6/mpich/3.1.4/nehalem/gcc49/ether/source/test'</div>
<div style="color: rgb(0, 0, 0);">(NOXMLCLOSE=YES && export NOXMLCLOSE && cd mpi && make testing)</div>
<div style="color: rgb(0, 0, 0);">make[2]: Entering directory `/gpfs22/local/centos6/mpich/3.1.4/nehalem/gcc49/ether/source/test/mpi'</div>
<div style="color: rgb(0, 0, 0);">./runtests -srcdir=. -tests=testlist \</div>
<div style="color: rgb(0, 0, 0);"><span class="Apple-tab-span" style="white-space: pre;"></span><span class="Apple-tab-span" style="white-space: pre;"></span>-mpiexec=/usr/local/mpich/3.1.4/nehalem/gcc49/ether/bin/mpiexec -xmlfile=summary.xml \</div>
<div style="color: rgb(0, 0, 0);"><span class="Apple-tab-span" style="white-space: pre;"></span><span class="Apple-tab-span" style="white-space: pre;"></span>-tapfile=summary.tap -junitfile=summary.junit.xml</div>
<div style="color: rgb(0, 0, 0);">Looking in ./testlist</div>
<div style="color: rgb(0, 0, 0);">Processing directory attr</div>
<div style="color: rgb(0, 0, 0);">Looking in ./attr/testlist</div>
<div style="color: rgb(0, 0, 0);">Failed to build attrt; make[3]: Entering directory `/gpfs22/local/centos6/mpich/3.1.4/nehalem/gcc49/ether/source/test/mpi/attr'</div>
<div style="color: rgb(0, 0, 0);"> CC attrt.o</div>
<div style="color: rgb(0, 0, 0);"> CCLD attrt</div>
<div style="color: rgb(0, 0, 0);">/usr/bin/ld: cannot find -lpmi</div>
<div style="color: rgb(0, 0, 0);">collect2: ld returned 1 exit status</div>
<div style="color: rgb(0, 0, 0);">make[3]: *** [attrt] Error 1</div>
<div style="color: rgb(0, 0, 0);">make[3]: Leaving directory `/gpfs22/local/centos6/mpich/3.1.4/nehalem/gcc49/ether/source/test/mpi/attr'</div>
<div style="color: rgb(0, 0, 0);"><br>
</div>
<div style="color: rgb(0, 0, 0);"><br>
</div>
<div style="color: rgb(0, 0, 0);">although /usr/scheduler/slurm/lib/libpmi.la exists.</div>
<div style="color: rgb(0, 0, 0);">So I tried to explicitly use the lib and include path in the configure options: no luck.</div>
<div style="color: rgb(0, 0, 0);">I also set "-L/usr/scheduler/slurm/lib" in the xFLAGS, but no effect.</div>
<div style="color: rgb(0, 0, 0);">Any suggestion?</div>
<div style="color: rgb(0, 0, 0);">Please let me know if I missed some information and thank you in advance for your help.</div>
<div style="color: rgb(0, 0, 0);"><br>
</div>
<div style="color: rgb(0, 0, 0);">Best regards,</div>
<div style="color: rgb(0, 0, 0);">Davide</div>
<div style="color: rgb(0, 0, 0);"><br>
</div>
<div style="color: rgb(0, 0, 0);">-- </div>
<div class="-x-evo-signature-wrapper" style="color: rgb(0, 0, 0);">
<div>Davide Vanzo, PhD<br>
Application Developer<br>
Advanced Computing Center for Research and Education (ACCRE)<br>
Vanderbilt University - Hill Center 201<br>
<a href="http://www.accre.vanderbilt.edu/" rel="noreferrer" target="_blank">www.accre.vanderbilt.edu</a></div>
</div>
</div>
</div>
</div>
</span>
</body>
</html>