<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif;">
<div>Good Day All,</div>
<div><br>
</div>
<div>We’re trying to get a build of 3.2 for our centos7 hpc environment using IB. we don’t have mxm installed so I’m trying this:</div>
<div><br>
</div>
<div>./configure --prefix=/software/tools/apps/mpich/gnu/3.2 --with-pm=hydra -with-device=ch3:nemesis --with-ibverbs=/usr --with-pbs=/opt/torque</div>
<div><br>
</div>
<div>But we end up with backtraces of the form:</div>
<div><br>
</div>
<div> <span style="color: inherit; font-family: inherit; font-size: inherit; white-space: pre-wrap;">mpi_script_launcher.run:17440 terminated with signal 7 at PC=7f8caca067f4 SP=7fff7c4ffb20. Backtrace:</span></div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;"></pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">mpi_script_launcher.run:17441 terminated with signal 7 at PC=7fece77297f4 SP=7ffd3b5ff100. Backtrace:</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">/software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPID_nem_init+0x964)[0x7f8caca067f4]</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">/software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPID_nem_init+0x964)[0x7fece77297f4]</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">/software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPIDI_CH3_Init+0x29)[0x7f8cac9f7609]</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">/software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPIDI_CH3_Init+0x29)[0x7fece771a609]</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">/software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPID_Init+0x18b)[0x7f8cac9ececb]</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">/software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPID_Init+0x18b)[0x7fece770fecb]</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">/software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPIR_Init_thread+0x34c)[0x7f8cac95345c]</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">/software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPIR_Init_thread+0x34c)[0x7fece767645c]</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">/software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPI_Init+0x7e)[0x7f8cac952ede]</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">/home/m8a/mpi_script_launcher/mpi_script_launcher.run[0x4008d0]</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">/software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPI_Init+0x7e)[0x7fece7675ede]</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">/home/m8a/mpi_script_launcher/mpi_script_launcher.run[0x4008d0]</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f8cac4fcb15]</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">/home/m8a/mpi_script_launcher/mpi_script_launcher.run[0x4007c9]</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">/lib64/libc.so.6(__libc_start_main+0xf5)[0x7fece721fb15]</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">/home/m8a/mpi_script_launcher/mpi_script_launcher.run[0x4007c9]</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;"><br></pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;"><br></pre>
</div>
Similarly with mvapich we get failures of this form:
<div>
<div id="MAC_OUTLOOK_SIGNATURE"></div>
</div>
<div><br>
</div>
<div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; padding-top: 0.5rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">mpirun -n 4 /home/m8a/mpi_script_launcher/MVAPICH/mpi_script_launcher.run /home/m8a/mpi_script_launcher/mpi_bash_script_example.sh </pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">[cli_0]: aborting job:</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">Fatal error in MPI_Init:</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">Other MPI error, error stack:</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">MPIR_Init_thread(514)..........: </pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">MPID_Init(365).................: channel initialization failed</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">MPIDI_CH3_Init(495)............: </pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">MPIDI_CH3I_SHMEM_Helper_fn(921): write: Success</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;"></pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;"></pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">===================================================================================</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">= PID 17901 RUNNING AT mod-condo-login02.ornl.gov</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">= EXIT CODE: 1</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">= CLEANING UP REMAINING PROCESSES</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">= YOU CAN IGNORE THE BELOW CLEANUP MESSAGES</pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;">===================================================================================</pre>
</div>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;"><br></pre>
</div>
<div style="box-sizing: border-box; counter-increment: snippet 1; position: relative; margin-left: 40px; padding-right: 0.625rem; color: rgb(85, 84, 89); font-family: monospace; font-size: 13px;">
<pre style="box-sizing: border-box; padding: 0px; font-size: inherit; color: inherit; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; display: inline-block; margin-top: 0px; margin-bottom: 0px; line-height: inherit; word-break: normal; word-wrap: break-word; white-space: pre-wrap; border: 0px solid rgba(0, 0, 0, 0.14902); z-index: 2; position: relative; overflow: visible; max-width: 100%; min-height: 0.8rem; text-rendering: auto; font-family: inherit !important; background-position: 0px 0px;"><br></pre>
</div>
I suspect we are doing something silly here but I’m not sure what, and openmpi code on the same cluster runs fine.
<div><br>
</div>
<div>Is there a current recommendation for IB/pbs/torque build flags?</div>
<div><br>
</div>
<div>— michael</div>
<div><br>
</div>
</body>
</html>