<div><div dir="auto">There is no way for internode latency to be less than ~1 us. Thus, I conclude you measure intranode results with MPICH and internode with everything else.</div></div><div dir="auto"><br></div><div dir="auto">Check your mpirun commands and try again. </div><div dir="auto"><br></div><div dir="auto">Jeff</div><div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Nov 26, 2019 at 10:57 AM Sajid Ali via discuss <<a href="mailto:discuss@mpich.org">discuss@mpich.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">
<p style="margin:0px 0px 1.2em">Hi MPICH-developers, </p>
<p style="margin:0px 0px 1.2em">I’ve been trying to benchmark
mpi software currently available on my univ. cluster with OSU micro benchmarks. I
used this script to build and run the benchmark : <code style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;margin:0px 0.15em;padding:0px 0.3em;white-space:pre-wrap;border:1px solid rgb(234,234,234);background-color:rgb(248,248,248);border-radius:3px;display:inline"><a href="https://github.com/s-sajid-ali/nu-quest-mpi/blob/master/curr_mpi_bench/submit.sh" target="_blank">https://github.com/s-sajid-ali/nu-quest-mpi/blob/master/curr_mpi_bench/submit.sh</a></code></p>
<p style="margin:0px 0px 1.2em">And here is a plot of the results I obtained :<br><img src="cid:ii_k3dgygab0" alt="curr_mpi.png" style="width:667px;max-width:100%"></p><p style="margin:0px 0px 1.2em">Could
someone give me a clue as to why the behaviour of mpich bandwidth vs
latency is so odd ? Why does the bandwidth increase first and then fall
off at high message sizes? </p>
<p style="margin:0px 0px 1.2em">I’m also attaching the
config.log for this particular mpich build which seems to indicate that
MPICH was built with ch3 and nemesis:tcp. Does this mean that it didn’t
pick up the infiniband libraries or that it is somehow using IPoIB? Is
there a way to get this information ?</p>
<p style="margin:0px 0px 1.2em">Thanks in advance for the help! <br><br><br>--<br>Sajid Ali | PhD Candidate<br>Applied Physics<br>Northwestern University<br><a href="http://s-sajid-ali.github.io" target="_blank">s-sajid-ali.github.io</a></p>
</div>
_______________________________________________<br>
discuss mailing list <a href="mailto:discuss@mpich.org" target="_blank">discuss@mpich.org</a><br>
To manage subscription options or unsubscribe:<br>
<a href="https://lists.mpich.org/mailman/listinfo/discuss" rel="noreferrer" target="_blank">https://lists.mpich.org/mailman/listinfo/discuss</a><br>
</blockquote></div></div>-- <br><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature">Jeff Hammond<br><a href="mailto:jeff.science@gmail.com" target="_blank">jeff.science@gmail.com</a><br><a href="http://jeffhammond.github.io/" target="_blank">http://jeffhammond.github.io/</a></div>