<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body text="#000000" bgcolor="#FFFFFF">
Hi Abu,<br>
<br>
Thanks for reporting this. Can you please try the latest release
with ch3/sock and see if you still have this error ? <br>
<br>
Min<br>
<div class="moz-cite-prefix">On 2018/07/01 21:47, Abu Naser wrote:<br>
</div>
<blockquote type="cite" cite="mid:BLUPR0501MB2003414CB97CA97A0242D0BC97430@BLUPR0501MB2003.namprd05.prod.outlook.com">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr">
<div id="divtagdefaultwrapper" style="" dir="ltr">
<p style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 12pt; margin-top: 0px;
margin-bottom: 0px;">
Hello Min,</p>
<p style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 12pt; margin-top: 0px;
margin-bottom: 0px;">
<br>
</p>
<p style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 12pt;">
After compiling my mpich-3.2.1 with sock, while I was trying
to run any program including osu benchmark or examples/cpi
in two machines, I have received following error -</p>
<p style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 12pt;">
<br>
</p>
<div style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 16px;">
<i><span style="font-size: 10pt;">Process 3 of 4 is on
dhcp16194</span></i></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 16px;">
<i><span style="font-size: 10pt;">Process 1 of 4 is on
dhcp16194</span></i></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 16px;">
<i><span style="font-size: 10pt;">Process 0 of 4 is on
dhcp16198</span></i></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 16px;">
<i><span style="font-size: 10pt;">Process 2 of 4 is on
dhcp16198</span></i></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 16px;">
<i><span style="font-size: 10pt;">Fatal error in PMPI_Bcast:
Unknown error class, error stack:</span></i></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 16px;">
<i><span style="font-size: 10pt;">PMPI_Bcast(1600)............................:
MPI_Bcast(buf=0x7ffc1808542c, count=1, MPI_INT, root=0,
MPI_COMM_WORLD) failed</span></i></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 16px;">
<i><span style="font-size: 10pt;">MPIR_Bcast_impl(1452).......................: </span></i></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 16px;">
<i><span style="font-size: 10pt;">MPIR_Bcast(1476)............................: </span></i></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 16px;">
<i><span style="font-size: 10pt;">MPIR_Bcast_intra(1249)......................: </span></i></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 16px;">
<i><span style="font-size: 10pt;">MPIR_SMP_Bcast(1081)........................: </span></i></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 16px;">
<i><span style="font-size: 10pt;">MPIR_Bcast_binomial(285)....................: </span></i></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 16px;">
<i><span style="font-size: 10pt;">MPIC_Send(303)..............................: </span></i></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 16px;">
<i><span style="font-size: 10pt;">MPIC_Wait(226)..............................: </span></i></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 16px;">
<i><span style="font-size: 10pt;">MPIDI_CH3i_Progress_wait(242)...............:
an error occurred while handling an event returned by
MPIDU_Sock_Wait()</span></i></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 16px;">
<i><span style="font-size: 10pt;">MPIDI_CH3I_Progress_handle_sock_event(698)..: </span></i></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 16px;">
<i><span style="font-size: 10pt;">MPIDI_CH3_Sockconn_handle_connect_event(597):
[ch3:sock] failed to connnect to remote process</span></i></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 16px;">
<i><span style="font-size: 10pt;">MPIDU_Socki_handle_connect(808).............:
connection failure (set=0,sock=1,errno=111:Connection
refused)</span></i></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 16px;">
<i><span style="font-size: 10pt;">MPIR_SMP_Bcast(1088)........................: </span></i></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 16px;">
<i><span style="font-size: 10pt;">MPIR_Bcast_binomial(310)....................:
Failure during collective</span></i></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 16px;">
<i><span style="font-size: 10pt;">Fatal error in PMPI_Bcast:
Other MPI error, error stack:</span></i></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 16px;">
<i><span style="font-size: 10pt;">PMPI_Bcast(1600)........:
MPI_Bcast(buf=0x7ffd9eeebdac, count=1, MPI_INT, root=0,
MPI_COMM_WORLD) failed</span></i></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 16px;">
<i><span style="font-size: 10pt;">MPIR_Bcast_impl(1452)...: </span></i></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 16px;">
<i><span style="font-size: 10pt;">MPIR_Bcast(1476)........: </span></i></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 16px;">
<i><span style="font-size: 10pt;">MPIR_Bcast_intra(1249)..: </span></i></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 16px;">
<i><span style="font-size: 10pt;">MPIR_SMP_Bcast(1088)....: </span></i></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 16px;">
<i><span style="font-size: 10pt;">MPIR_Bcast_binomial(310):
Failure during collective</span></i></div>
<br style="font-family: Calibri, Helvetica, sans-serif,
EmojiFont, "Apple Color Emoji", "Segoe UI
Emoji", NotoColorEmoji, "Segoe UI Symbol",
"Android Emoji", EmojiSymbols; font-size: 16px;">
<p style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 12pt;">
</p>
<p style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 16px;">
<span style="font-size: 12pt;">I checked the mpich FAQ and
also mpich discussion list. Based on that I have checked </span>followings<span style="font-size: 12pt;"> </span><span style="font-size:
12pt;">and found they are fine in my machines -</span><br>
</p>
<p style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 12pt;">
<span style="font-size: 12pt;">- firewall is disabled in
both machine</span></p>
<p style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 12pt;">
<span style="font-size: 12pt;">- I can do </span>password
less<span style="font-size: 12pt;"> ssh in both machine</span></p>
<p style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 12pt;">
<span style="font-size: 12pt;">- /etc/hosts in both machine
configured with ip address and name properly</span></p>
<p style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 12pt;">
<span style="font-size: 12pt;">- I have updated the library
path and used absolute path for mpiexec</span></p>
<p style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 12pt;">
<span style="font-size: 12pt;">- Most importantly when I
configured and build mpich with tcp, it works fine.</span></p>
<p style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 12pt;">
<span style="font-size: 12pt;"><br>
</span></p>
<p style=""><span style="font-size: 12pt;"> I think I am </span><span style="font-size: 12pt;">missing something but could not
figure out yet. Any help would be
</span>appreciated<span style="font-size: 12pt;">.</span></p>
<p style=""><span style="font-size: 12pt;"><br>
</span></p>
<p style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 12pt;">
<span style="font-size: 12pt;">Thank you.</span></p>
<br>
<p style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 12pt; margin-top: 0px;
margin-bottom: 0px;">
<br>
</p>
<p style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 12pt; margin-top: 0px;
margin-bottom: 0px;">
<br>
</p>
<p style="color: rgb(0, 0, 0); font-family: Calibri,
Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 12pt; margin-top: 0px;
margin-bottom: 0px;">
<br>
</p>
<div id="Signature" style="color: rgb(0, 0, 0); font-family:
Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color
Emoji", "Segoe UI Emoji", NotoColorEmoji,
"Segoe UI Symbol", "Android Emoji",
EmojiSymbols; font-size: 12pt;">
<div id="divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:rgb(0,0,0);
font-family:Calibri,Helvetica,sans-serif,"EmojiFont","Apple
Color Emoji","Segoe UI
Emoji",NotoColorEmoji,"Segoe UI
Symbol","Android Emoji",EmojiSymbols">
<p><br>
</p>
<p align="left"><span style="font-size:10pt;
font-family:Calibri,Helvetica,sans-serif">Best
Regards,</span></p>
<span style="font-family:Calibri,Helvetica,sans-serif;
font-size:10pt"></span>
<div align="left"><span style="font-size:11pt;
font-family:Calibri,Helvetica,sans-serif"></span></div>
<span style="font-family:Calibri,Helvetica,sans-serif;
font-size:10pt"></span>
<p align="left"><span style="font-size:10pt;
font-family:Calibri,Helvetica,sans-serif">Abu Naser</span><br>
</p>
</div>
</div>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> Min
Si <a class="moz-txt-link-rfc2396E" href="mailto:msi@anl.gov"><msi@anl.gov></a><br>
<b>Sent:</b> Tuesday, June 26, 2018 12:54:29 PM<br>
<b>To:</b> <a class="moz-txt-link-abbreviated" href="mailto:discuss@mpich.org">discuss@mpich.org</a><br>
<b>Subject:</b> Re: [mpich-discuss] osu_latency test: why
8KB takes less time than 4KB and 2KB takes less time than
1KB?</font>
<div> </div>
</div>
<meta content="text/html; charset=Windows-1252">
<div style="background-color:#FFFFFF">Hi Abu,<br>
<br>
I think the results are stable enough. Perhaps you could also
try the following tests, and see if similar trend exists:<br>
- MPICH/socket (set `--with-device=ch3:sock` at configure)<br>
- A socket-based pingpong test without MPI. <br>
<br>
At this point, I could not think of any MPI-specific design
for 2k/8k messages. My guess is that it is related to your
network connection.
<br>
<br>
Min<br>
<br>
<div class="x_moz-cite-prefix">On 2018/06/24 11:09, Abu Naser
wrote:<br>
</div>
<blockquote type="cite">
<meta content="text/html; charset=Windows-1252">
<div id="x_divtagdefaultwrapper" dir="ltr">
<div id="x_divtagdefaultwrapper" dir="ltr">
<p>Hello Min and Jeff,</p>
<p><br>
</p>
<p>Here is my experiment results. Default number of
iterations in osu_latency for 0B – 8KB is 10,000. With
that setting I had run the osu_latency 100 times and
found standard deviation 33 for 8KB message size.</p>
<p><br>
</p>
<p>So later I have set the iteration to 50,000 and
100,000 for 1KB – 16KB message size. Then run
osu_latency for 100 times for each setting and take
the average and standard deviation.</p>
<p><br>
</p>
<table width="665">
<colgroup><col width="99"><col width="112"><col width="118"><col width="154"><col width="140"></colgroup>
<tbody>
<tr>
<td width="99">
<p><b>Msg Size in Bytes</b></p>
</td>
<td width="112">
<p><b>Avg time in us (50K iterations)</b></p>
</td>
<td width="118">
<p><b>Avg time in us (100k iterations)</b></p>
</td>
<td width="154">
<p><b>Standard deviation (50K iterations)</b></p>
</td>
<td width="140">
<p><b>Standard deviation (100K iterations)</b></p>
</td>
</tr>
<tr>
<td width="99">
<p>1k</p>
</td>
<td width="112">
<p>85.10</p>
</td>
<td width="118">
<p>84.9</p>
</td>
<td width="154">
<p>0.55</p>
</td>
<td width="140">
<p>0.45</p>
</td>
</tr>
<tr>
<td width="99">
<p>2k</p>
</td>
<td width="112">
<p>75.79</p>
</td>
<td width="118">
<p>74.63</p>
</td>
<td width="154">
<p>5.09</p>
</td>
<td width="140">
<p>4.44</p>
</td>
</tr>
<tr>
<td width="99">
<p>4k</p>
</td>
<td width="112">
<p>273.80</p>
</td>
<td width="118">
<p>274.71</p>
</td>
<td width="154">
<p>4.18</p>
</td>
<td width="140">
<p>2.45</p>
</td>
</tr>
<tr>
<td width="99">
<p>8k</p>
</td>
<td width="112">
<p>258.56</p>
</td>
<td width="118">
<p>249.83</p>
</td>
<td width="154">
<p>21.14</p>
</td>
<td width="140">
<p>28</p>
</td>
</tr>
<tr>
<td width="99" height="24">
<p>16k</p>
</td>
<td width="112">
<p>281.31</p>
</td>
<td width="118">
<p>281.02</p>
</td>
<td width="154">
<p>3.22</p>
</td>
<td width="140">
<p>4.10</p>
</td>
</tr>
</tbody>
</table>
<p><br>
</p>
<p><br>
</p>
<p>The standard deviation of 8K message is so high and
that implies it actually not producing any consistent
latency time. Looks like that's the reason for 8K is
taking less time than 4K.</p>
<p><br>
</p>
<p>Meanwhile, 2K has standard deviation less than 5 but
1K message latency timing are more densely populated
than 2K. So probably this is the explanation for 2K
message less latency time.</p>
<p><br>
</p>
<p>Thank you for your suggestions.</p>
<br>
<p><br>
</p>
<div id="x_Signature">
<div id="x_divtagdefaultwrapper" dir="ltr">
<p><br>
</p>
<p><span>Best Regards,</span></p>
<span></span>
<div><span></span></div>
<span></span>
<p><span>Abu Naser</span><br>
</p>
</div>
</div>
</div>
<hr tabindex="-1">
<div id="x_divRplyFwdMsg" dir="ltr"><b>From:</b> Abu Naser<br>
<b>Sent:</b> Wednesday, June 20, 2018 1:48:53 PM<br>
<b>To:</b> <a class="x_moz-txt-link-abbreviated
OWAAutoLink" href="mailto:discuss@mpich.org" id="LPlnk729146" previewremoved="true" moz-do-not-send="true">
discuss@mpich.org</a><br>
<b>Subject:</b> Re: [mpich-discuss] osu_latency test:
why 8KB takes less time than 4KB and 2KB takes less time
than 1KB?
<div> </div>
</div>
<meta content="text/html; charset=iso-8859-1">
<div dir="ltr">
<div id="x_x_divtagdefaultwrapper" dir="ltr">
<div id="x_x_divtagdefaultwrapper" dir="ltr">
<p>Hello Min,</p>
<p><br>
</p>
<p>Thanks for the clarification. I will do the
experiment.<br>
</p>
<p><br>
</p>
<div id="x_x_Signature">
<div id="x_x_divtagdefaultwrapper" dir="ltr">
<p>Thanks.</p>
<p><span>Best Regards,</span></p>
<span></span>
<div><span></span></div>
<span></span>
<p><span>Abu Naser</span><br>
</p>
</div>
</div>
</div>
<hr tabindex="-1">
<div id="x_x_divRplyFwdMsg" dir="ltr"><b>From:</b> Min
Si <a class="x_moz-txt-link-rfc2396E OWAAutoLink" href="mailto:msi@anl.gov" id="LPlnk558260" previewremoved="true" moz-do-not-send="true">
<msi@anl.gov></a><br>
<b>Sent:</b> Wednesday, June 20, 2018 1:39:30 PM<br>
<b>To:</b> <a class="x_moz-txt-link-abbreviated
OWAAutoLink" href="mailto:discuss@mpich.org" id="LPlnk472728" previewremoved="true" moz-do-not-send="true">
discuss@mpich.org</a><br>
<b>Subject:</b> Re: [mpich-discuss] osu_latency
test: why 8KB takes less time than 4KB and 2KB takes
less time than 1KB?
<div> </div>
</div>
<meta content="text/html; charset=Windows-1252">
<div>Hi Abu,<br>
<br>
I think Jeff means that you should run your
experiment with more iterations in order to get a
stable results.<br>
- Increase the iteration of for loop in each
execution (I think osu benchmark allows you to set
it)<br>
- Run the experiments 10 or 100 times, and take the
average and standard deviation.<br>
<br>
If you see a very small standard deviation (e.g.,
<=5%), then the trend is stable and you might not
see such gaps.<br>
<br>
Best regards,<br>
Min<br>
<div class="x_x_x_moz-cite-prefix">On 2018/06/20
12:14, Abu Naser wrote:<br>
</div>
<blockquote type="cite">
<div id="x_x_x_divtagdefaultwrapper" dir="ltr">
<p>Hello Jeff,</p>
<p><br>
</p>
<p>Yes, I am using a switch and other
machines are also connected with that switch.
<br>
</p>
<p>If I remove other machines and just use my
two node with the switch, then will it improve
the performance by 200 ~ 400 iterations?</p>
<p>Meanwhile I will give a try with a single
dedicated cable. <span></span><br>
</p>
<p><br>
</p>
<p>Thank you.<br>
</p>
<div id="x_x_x_Signature">
<div id="x_x_x_divtagdefaultwrapper" dir="ltr">
<p><br>
</p>
<p><span>Best Regards,</span></p>
<span></span>
<div><span></span></div>
<span></span>
<p><span>Abu Naser</span><br>
</p>
</div>
</div>
</div>
<hr tabindex="-1">
<div id="x_x_x_divRplyFwdMsg" dir="ltr"><b>From:</b>
Jeff Hammond <a class="x_x_x_moz-txt-link-rfc2396E
x_x_OWAAutoLink" href="mailto:jeff.science@gmail.com" id="LPlnk983157" previewremoved="true" moz-do-not-send="true">
<jeff.science@gmail.com></a><br>
<b>Sent:</b> Wednesday, June 20, 2018 12:52:06
PM<br>
<b>To:</b> MPICH<br>
<b>Subject:</b> Re: [mpich-discuss] osu_latency
test: why 8KB takes less time than 4KB and 2KB
takes less time than 1KB?
<div> </div>
</div>
<meta content="text/html; charset=utf-8">
<div>
<div dir="ltr">Is the ethernet connection a
single dedicated cable between the two
machines or are you running through a switch
that handles other traffic?
<div><br>
</div>
<div>My best guess is that this is noise and
that you may be able to avoid it by running
a very long time, e.g. 10000 iterations.</div>
<div><br>
</div>
<div>Jeff</div>
</div>
<div class="x_x_x_x_gmail_extra"><br>
<div class="x_x_x_x_gmail_quote">On Wed, Jun
20, 2018 at 6:53 AM, Abu Naser <span dir="ltr">
<<a href="mailto:an16e@my.fsu.edu" target="_blank" id="LPlnk305789" class="x_x_OWAAutoLink" previewremoved="true" moz-do-not-send="true">an16e@my.fsu.edu</a>></span>
wrote:<br>
<blockquote class="x_x_x_x_gmail_quote">
<div dir="ltr">
<div id="x_x_x_x_m_6077755676379859201divtagdefaultwrapper" dir="ltr">
<p><br>
</p>
<p>Good day to all,</p>
<p><br>
</p>
<p>I had run point to point
osu_latency test in two nodes for
200 times. Followings are the
average time in microsecond for
various size of the messages -</p>
<div>1KB 84.8514 us<br>
<span>2KB 73.52535</span> us<br>
4KB 272.55275 us<br>
<span>8KB 234.86385</span> us<br>
16KB 288.88 us<br>
32KB 523.3725 us<br>
64KB 910.4025 us</div>
<p><br>
</p>
<p>From the above looks like, 2KB
message has less latency than 1 KB
and 8KB has less latency than 4KB.
<br>
</p>
<p>I was looking for explanation of
this behavior but did not get any.</p>
<p><br>
</p>
<ol>
<li><span>MPIR_CVAR_CH3_EAGER_MAX_MSG_<wbr>SIZE</span><span>
is set to 128KB. So none of the
above message size is using
Rendezvous protocol. Is there
any partition inside eager
protocol (e.g. 0 - 512 bytes,
1KB - 8KB, 16KB - 64KB)? If yes
then what are the boundaries for
them? Can I log them with
debug-event-logging? </span><br>
</li>
</ol>
<p><br>
</p>
<p>Setup I am using:</p>
<p>- two nodes has intel core i7, one
with 16gb memory another one 8gb</p>
<p>- mpich 3.2.1, configured and build
to use nemesis tcp</p>
<p>- 1gb Ethernet connection</p>
<p>- NFS is using for sharing<br>
</p>
<p>- osu_latency : uses MPI_Send and
MPI_Recv</p>
<p>- <span>MPIR_CVAR_CH3_EAGER_MAX_MSG_<wbr>SIZE</span>=
<span>131072</span> (128KB)<br>
</p>
<p><br>
</p>
<p>Can anyone help me on that? Thanks
in advance.<br>
</p>
<p><br>
</p>
<p><br>
</p>
<div id="x_x_x_x_m_6077755676379859201Signature">
<div id="x_x_x_x_m_6077755676379859201divtagdefaultwrapper" dir="ltr">
<p><br>
</p>
<p><span>Best Regards,</span></p>
<span></span>
<div><span></span></div>
<span></span>
<p><span>Abu Naser</span><br>
</p>
</div>
</div>
</div>
</div>
<br>
______________________________<wbr>_________________<br>
discuss mailing list <a href="mailto:discuss@mpich.org" id="LPlnk816471" class="x_x_OWAAutoLink" previewremoved="true" moz-do-not-send="true">discuss@mpich.org</a><br>
To manage subscription options or
unsubscribe:<br>
<a href="https://lists.mpich.org/mailman/listinfo/discuss" rel="noreferrer" target="_blank" id="LPlnk624595" class="x_x_OWAAutoLink" previewremoved="true" moz-do-not-send="true">https://lists.mpich.org/<wbr>mailman/listinfo/discuss</a><br>
<br>
</blockquote>
</div>
<br>
<br>
<div><br>
</div>
-- <br>
<div class="x_x_x_x_gmail_signature">Jeff
Hammond<br>
<a href="mailto:jeff.science@gmail.com" target="_blank" id="LPlnk314993" class="x_x_OWAAutoLink" previewremoved="true" moz-do-not-send="true">jeff.science@gmail.com</a><br>
<a href="http://jeffhammond.github.io/" target="_blank" id="LPlnk861434" class="x_x_OWAAutoLink" previewremoved="true" moz-do-not-send="true">http://jeffhammond.github.io/</a></div>
</div>
</div>
<br>
<fieldset class="x_x_x_mimeAttachmentHeader"></fieldset>
<br>
<pre>_______________________________________________
discuss mailing list <a class="x_x_x_moz-txt-link-abbreviated x_x_OWAAutoLink" href="mailto:discuss@mpich.org" id="LPlnk657371" previewremoved="true" moz-do-not-send="true">discuss@mpich.org</a>
To manage subscription options or unsubscribe:
<a class="x_x_x_moz-txt-link-freetext x_x_OWAAutoLink" href="https://lists.mpich.org/mailman/listinfo/discuss" id="LPlnk669988" previewremoved="true" moz-do-not-send="true">https://lists.mpich.org/mailman/listinfo/discuss</a>
</pre>
</blockquote>
<br>
</div>
</div>
</div>
</div>
<br>
<fieldset class="x_mimeAttachmentHeader"></fieldset>
<br>
<pre>_______________________________________________
discuss mailing list <a class="x_moz-txt-link-abbreviated OWAAutoLink" href="mailto:discuss@mpich.org" id="LPlnk832953" previewremoved="true" moz-do-not-send="true">discuss@mpich.org</a>
To manage subscription options or unsubscribe:
<a class="x_moz-txt-link-freetext OWAAutoLink" href="https://lists.mpich.org/mailman/listinfo/discuss" id="LPlnk481779" previewremoved="true" moz-do-not-send="true">https://lists.mpich.org/mailman/listinfo/discuss</a>
</pre>
</blockquote>
<br>
</div>
</div>
<!--'"--><br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
discuss mailing list <a class="moz-txt-link-abbreviated" href="mailto:discuss@mpich.org">discuss@mpich.org</a>
To manage subscription options or unsubscribe:
<a class="moz-txt-link-freetext" href="https://lists.mpich.org/mailman/listinfo/discuss">https://lists.mpich.org/mailman/listinfo/discuss</a>
</pre>
</blockquote>
<br>
</body>
</html>