<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hi Feimi,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hydra should be able to work with slurm. How are you launching the job and what is the failure message?</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
-- <br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hui Zhou<br>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Feimi Yu via discuss <discuss@mpich.org><br>
<b>Sent:</b> Friday, September 17, 2021 10:55 AM<br>
<b>To:</b> discuss@mpich.org <discuss@mpich.org><br>
<b>Cc:</b> Feimi Yu <yuf2@rpi.edu><br>
<b>Subject:</b> [mpich-discuss] Installing MPICH on clusters</font>
<div> </div>
</div>
<div>
<p>Hi,</p>
<p>I'm working on a supercomputer which only provides Spectrum MPI implementation in modules. Since our code does not perform well with Spectrum MPI I decided to install an MPICH build on our own partition (I'm not an administrator.) The supercomputer has a
rhel8 system on ppc64le architecture with Slurm as the process manager. I tried several building options according to the user guide but could not run a job so I have a few questions. Here are things I tried:</p>
<p>1. Build with Hydra PM. I could not launch a job with Hydra at all.</p>
<p>2. Then I decided to use ``--with-pm=none`` option to build and use srun + ``mpiexec -f hostfile`` to launch my job. But what confuses me is the PMI setting:</p>
<p>srun --mpi=list gives following:</p>
<p>srun: mpi/mpichgm<br>
srun: mpi/mpichmx<br>
srun: mpi/none<br>
srun: mpi/mvapich<br>
srun: mpi/openmpi<br>
srun: mpi/pmi2<br>
srun: mpi/lam<br>
srun: mpi/mpich1_p4<br>
srun: mpi/mpich1_shmem</p>
<p>At first I tried use pmix since I found pmix libraries. But it didn't do the trick. It segfaults on PMPI_Init_thread(). The error message is:</p>
<p><i>[dcs135:2312190] PMIX ERROR: NOT-FOUND in file client/pmix_client.c at line 562</i></p>
<p><i>Abort(1090831) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: Other MPI error, error stack:</i><i><br>
</i><i>MPIR_Init_thread(159): </i><i><br>
</i><i>MPID_Init(509).......: </i><i><br>
</i><i>MPIR_pmi_init(92)....: PMIX_Init returned -46 </i><i><br>
</i><i>[dcs135:2312190:0:2312190] Caught signal 11 (Segmentation fault: address not mapped to object at address (nil))</i><i><br>
</i></p>
<p>Then I switched to pmi2 but make keeps telling me undefined reference to PMI2 library. (actually I couldn't find the pmi2 libraries either.)</p>
<p>Then I used ``--with-pmi=slurm``, and it turned out that I couldn't locate the Slurm header files. I guess I don't have the permission to access them.</p>
<p>I was wondering if it is still possible for me to build a usable MPICH as a user? If yes, how can I do to have the PMI work?<br>
</p>
<p>Thanks!</p>
Feimi </div>
</body>
</html>