<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    Could you please try mpich-3.3b3 ?<br>
    <a class="moz-txt-link-freetext" href="http://www.mpich.org/static/downloads/3.3b3/mpich-3.3b3.tar.gz">http://www.mpich.org/static/downloads/3.3b3/mpich-3.3b3.tar.gz</a><br>
    <br>
    Min<br>
    <div class="moz-cite-prefix">On 2018/07/02 13:01, Abu Naser wrote:<br>
    </div>
    <blockquote type="cite" cite="mid:BLUPR0501MB2003DCD7FDB382061050A6B997430@BLUPR0501MB2003.namprd05.prod.outlook.com">
      
      <style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
      <div id="divtagdefaultwrapper" style="font-size: 12pt; color:
        rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif,
        "EmojiFont", "Apple Color Emoji",
        "Segoe UI Emoji", NotoColorEmoji, "Segoe UI
        Symbol", "Android Emoji", EmojiSymbols;" dir="ltr">
        <div id="divtagdefaultwrapper" style="font-size: 12pt; color:
          rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif,
          "EmojiFont", "Apple Color Emoji",
          "Segoe UI Emoji", NotoColorEmoji, "Segoe UI
          Symbol", "Android Emoji", EmojiSymbols;" dir="ltr">
          <p style="margin-top:0;margin-bottom:0">Hello Min,</p>
          <p style="margin-top:0;margin-bottom:0"><br>
          </p>
          <p style="margin-top:0;margin-bottom:0">I have downloaded it
            from <a href="http://www.mpich.org/static/downloads/3.2.1/mpich-3.2.1.tar.gz" class="OWAAutoLink" id="LPlnk943697" previewremoved="true" moz-do-not-send="true">
http://www.mpich.org/static/downloads/3.2.1/mpich-3.2.1.tar.gz</a> but
            it did not work. I have received almost same error. Except
            this time no process information from my remote machine.</p>
          <p style="margin-top:0;margin-bottom:0"><b>Previously I have
              received this - </b>
            <br>
          </p>
          <div style=""><i><span style="font-size: 10pt; color: rgb(255,
                0, 0);">Process 3 of 4 is on dhcp16194</span></i></div>
          <div style=""><i><span style="font-size: 10pt; color: rgb(255,
                0, 0);">Process 1 of 4 is on dhcp16194</span></i></div>
          <div style=""><i><span style="font-size: 10pt; color: rgb(255,
                0, 0);">Process 0 of 4 is on dhcp16198</span></i></div>
          <div style=""><i><span style="font-size: 10pt; color: rgb(255,
                0, 0);">Process 2 of 4 is on dhcp16198</span></i></div>
          <p style="margin-top:0;margin-bottom:0"><b>With the new source
              code -</b></p>
          <div><i><span style="font-size: 10pt; color: rgb(255, 0, 0);">Process
                0 of 4 is on dhcp16198</span></i></div>
          <div><i><span style="font-size: 10pt; color: rgb(255, 0, 0);">Process
                2 of 4 is on dhcp16198</span></i></div>
          <p style="margin-top:0;margin-bottom:0"><br>
          </p>
          <p style="margin-top:0;margin-bottom:0"><b>Entire error
              message is:</b></p>
          <div><i><span style="font-size: 10pt;">Process 0 of 4 is on
                dhcp16198</span></i></div>
          <div><i><span style="font-size: 10pt;">Process 2 of 4 is on
                dhcp16198</span></i></div>
          <div><i><span style="font-size: 10pt;">Fatal error in
                PMPI_Bcast: Unknown error class, error stack:</span></i></div>
          <div><i><span style="font-size: 10pt;">PMPI_Bcast(1600)............................:
                MPI_Bcast(buf=0x7ffd1ee145f0, count=1, MPI_INT, root=0,
                MPI_COMM_WORLD) failed</span></i></div>
          <div><i><span style="font-size: 10pt;">MPIR_Bcast_impl(1452).......................: </span></i></div>
          <div><i><span style="font-size: 10pt;">MPIR_Bcast(1476)............................: </span></i></div>
          <div><i><span style="font-size: 10pt;">MPIR_Bcast_intra(1249)......................: </span></i></div>
          <div><i><span style="font-size: 10pt;">MPIR_SMP_Bcast(1081)........................: </span></i></div>
          <div><i><span style="font-size: 10pt;">MPIR_Bcast_binomial(285)....................: </span></i></div>
          <div><i><span style="font-size: 10pt;">MPIC_Send(303)..............................: </span></i></div>
          <div><i><span style="font-size: 10pt;">MPIC_Wait(226)..............................: </span></i></div>
          <div><i><span style="font-size: 10pt;">MPIDI_CH3i_Progress_wait(242)...............:
                an error occurred while handling an event returned by
                MPIDU_Sock_Wait()</span></i></div>
          <div><i><span style="font-size: 10pt;">MPIDI_CH3I_Progress_handle_sock_event(698)..: </span></i></div>
          <div><i><span style="font-size: 10pt;">MPIDI_CH3_Sockconn_handle_connect_event(597):
                [ch3:sock] failed to connnect to remote process</span></i></div>
          <div><i><span style="font-size: 10pt;">MPIDU_Socki_handle_connect(808).............:
                connection failure (set=0,sock=1,errno=111:Connection
                refused)</span></i></div>
          <div><i><span style="font-size: 10pt;">MPIR_SMP_Bcast(1088)........................: </span></i></div>
          <div><i><span style="font-size: 10pt;">MPIR_Bcast_binomial(310)....................:
                Failure during collective</span></i></div>
          <div><i><span style="font-size: 10pt;">Fatal error in
                PMPI_Bcast: Other MPI error, error stack:</span></i></div>
          <div><i><span style="font-size: 10pt;">PMPI_Bcast(1600)........:
                MPI_Bcast(buf=0x7ffe2eeb90f0, count=1, MPI_INT, root=0,
                MPI_COMM_WORLD) failed</span></i></div>
          <div><i><span style="font-size: 10pt;">MPIR_Bcast_impl(1452)...: </span></i></div>
          <div><i><span style="font-size: 10pt;">MPIR_Bcast(1476)........: </span></i></div>
          <div><i><span style="font-size: 10pt;">MPIR_Bcast_intra(1249)..: </span></i></div>
          <div><i><span style="font-size: 10pt;">MPIR_SMP_Bcast(1088)....: </span></i></div>
          <div><i><span style="font-size: 10pt;">MPIR_Bcast_binomial(310):
                Failure during collective</span></i></div>
          <br>
          <p style="margin-top:0;margin-bottom:0">Again if I configure
            the new source with tcp, it works fine.</p>
          <p style="margin-top:0;margin-bottom:0"><br>
          </p>
          <p style="margin-top:0;margin-bottom:0">Thank You.<br>
          </p>
          <div id="Signature">
            <div id="divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:rgb(0,0,0);
font-family:Calibri,Helvetica,sans-serif,"EmojiFont","Apple
              Color Emoji","Segoe UI
              Emoji",NotoColorEmoji,"Segoe UI
              Symbol","Android Emoji",EmojiSymbols">
              <p><br>
              </p>
              <p align="left"><span style="font-size:10pt;
                  font-family:Calibri,Helvetica,sans-serif">Best
                  Regards,</span></p>
              <span style="font-family:Calibri,Helvetica,sans-serif;
                font-size:10pt"></span>
              <div align="left"><span style="font-size:11pt;
                  font-family:Calibri,Helvetica,sans-serif"></span></div>
              <span style="font-family:Calibri,Helvetica,sans-serif;
                font-size:10pt"></span>
              <p align="left"><span style="font-size:10pt;
                  font-family:Calibri,Helvetica,sans-serif">Abu Naser</span><br>
              </p>
            </div>
          </div>
        </div>
        <hr style="display:inline-block;width:98%" tabindex="-1">
        <div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> Min
            Si <a class="moz-txt-link-rfc2396E" href="mailto:msi@anl.gov"><msi@anl.gov></a><br>
            <b>Sent:</b> Monday, July 2, 2018 11:56:51 AM<br>
            <b>To:</b> <a class="moz-txt-link-abbreviated" href="mailto:discuss@mpich.org">discuss@mpich.org</a><br>
            <b>Subject:</b> Re: [mpich-discuss] osu_latency test: why
            8KB takes less time than 4KB and 2KB takes less time than
            1KB?</font>
          <div> </div>
        </div>
        <meta content="text/html; charset=Windows-1252">
        <div style="background-color:#FFFFFF">Hi Abu,<br>
          <br>
          Thanks for reporting this. Can you please try the latest
          release with ch3/sock and see if you still have this error ?
          <br>
          <br>
          Min<br>
          <div class="x_moz-cite-prefix">On 2018/07/01 21:47, Abu Naser
            wrote:<br>
          </div>
          <blockquote type="cite">
            <div id="x_divtagdefaultwrapper" dir="ltr" style="font-size:
              12pt; color: rgb(0, 0, 0); font-family: Calibri,
              Helvetica, sans-serif, "EmojiFont", "Apple
              Color Emoji", "Segoe UI Emoji",
              NotoColorEmoji, "Segoe UI Symbol", "Android
              Emoji", EmojiSymbols;">
              <div id="x_divtagdefaultwrapper" dir="ltr" style="">
                <p style="">Hello Min,</p>
                <p style=""><br>
                </p>
                <p style="">After compiling my mpich-3.2.1 with sock,
                  while I was trying to run  any program including
                  osu benchmark or examples/cpi  in two machines, I have
                  received following error -</p>
                <p style=""><br>
                </p>
                <div style=""><i><span style="font-size:10pt">Process 3
                      of 4 is on dhcp16194</span></i></div>
                <div style=""><i><span style="font-size:10pt">Process 1
                      of 4 is on dhcp16194</span></i></div>
                <div style=""><i><span style="font-size:10pt">Process 0
                      of 4 is on dhcp16198</span></i></div>
                <div style=""><i><span style="font-size:10pt">Process 2
                      of 4 is on dhcp16198</span></i></div>
                <div style=""><i><span style="font-size:10pt">Fatal
                      error in PMPI_Bcast: Unknown error class, error
                      stack:</span></i></div>
                <div style=""><i><span style="font-size:10pt">PMPI_Bcast(1600)............................:
                      MPI_Bcast(buf=0x7ffc1808542c, count=1, MPI_INT,
                      root=0, MPI_COMM_WORLD) failed</span></i></div>
                <div style=""><i><span style="font-size:10pt">MPIR_Bcast_impl(1452).......................: </span></i></div>
                <div style=""><i><span style="font-size:10pt">MPIR_Bcast(1476)............................: </span></i></div>
                <div style=""><i><span style="font-size:10pt">MPIR_Bcast_intra(1249)......................: </span></i></div>
                <div style=""><i><span style="font-size:10pt">MPIR_SMP_Bcast(1081)........................: </span></i></div>
                <div style=""><i><span style="font-size:10pt">MPIR_Bcast_binomial(285)....................: </span></i></div>
                <div style=""><i><span style="font-size:10pt">MPIC_Send(303)..............................: </span></i></div>
                <div style=""><i><span style="font-size:10pt">MPIC_Wait(226)..............................: </span></i></div>
                <div style=""><i><span style="font-size:10pt">MPIDI_CH3i_Progress_wait(242)...............:
                      an error occurred while handling an event returned
                      by MPIDU_Sock_Wait()</span></i></div>
                <div style=""><i><span style="font-size:10pt">MPIDI_CH3I_Progress_handle_sock_event(698)..: </span></i></div>
                <div style=""><i><span style="font-size:10pt">MPIDI_CH3_Sockconn_handle_connect_event(597):
                      [ch3:sock] failed to connnect to remote process</span></i></div>
                <div style=""><i><span style="font-size:10pt">MPIDU_Socki_handle_connect(808).............:
                      connection failure
                      (set=0,sock=1,errno=111:Connection refused)</span></i></div>
                <div style=""><i><span style="font-size:10pt">MPIR_SMP_Bcast(1088)........................: </span></i></div>
                <div style=""><i><span style="font-size:10pt">MPIR_Bcast_binomial(310)....................:
                      Failure during collective</span></i></div>
                <div style=""><i><span style="font-size:10pt">Fatal
                      error in PMPI_Bcast: Other MPI error, error stack:</span></i></div>
                <div style=""><i><span style="font-size:10pt">PMPI_Bcast(1600)........:
                      MPI_Bcast(buf=0x7ffd9eeebdac, count=1, MPI_INT,
                      root=0, MPI_COMM_WORLD) failed</span></i></div>
                <div style=""><i><span style="font-size:10pt">MPIR_Bcast_impl(1452)...: </span></i></div>
                <div style=""><i><span style="font-size:10pt">MPIR_Bcast(1476)........: </span></i></div>
                <div style=""><i><span style="font-size:10pt">MPIR_Bcast_intra(1249)..: </span></i></div>
                <div style=""><i><span style="font-size:10pt">MPIR_SMP_Bcast(1088)....: </span></i></div>
                <div style=""><i><span style="font-size:10pt">MPIR_Bcast_binomial(310):
                      Failure during collective</span></i></div>
                <br style="">
                <p style=""><span style="font-size:12pt">I checked the
                    mpich FAQ and also mpich discussion list. Based on
                    that I have checked </span>followings<span style="font-size:12pt"> </span><span style="font-size:12pt">and found  they are fine in
                    my machines -</span><br>
                </p>
                <p style=""><span style="font-size:12pt">- firewall is
                    disabled in both machine</span></p>
                <p style=""><span style="font-size:12pt">- I can do </span>password
                  less<span style="font-size:12pt"> ssh in both machine</span></p>
                <p style=""><span style="font-size:12pt">- /etc/hosts in
                    both machine configured with ip address and name
                    properly</span></p>
                <p style=""><span style="font-size:12pt">- I have
                    updated the library path and used absolute path for
                    mpiexec</span></p>
                <p style=""><span style="font-size:12pt">- Most
                    importantly when I configured and build mpich with
                    tcp, it works fine.</span></p>
                <p style=""><span style="font-size:12pt"><br>
                  </span></p>
                <p style=""><span style="font-size:12pt"> I think I am </span><span style="font-size:12pt">missing something but could
                    not figure out yet. Any help would be
                  </span>appreciated<span style="font-size:12pt">.</span></p>
                <p style=""><span style="font-size:12pt"><br>
                  </span></p>
                <p style=""><span style="font-size:12pt">Thank you.</span></p>
                <br>
                <p style=""><br>
                </p>
                <p style=""><br>
                </p>
                <p style=""><br>
                </p>
                <div id="x_Signature" style="">
                  <div id="x_divtagdefaultwrapper" dir="ltr" style="">
                    <p><br>
                    </p>
                    <p align="left"><span style="font-size:10pt;
                        font-family:Calibri,Helvetica,sans-serif">Best
                        Regards,</span></p>
                    <span style="font-family:Calibri,Helvetica,sans-serif;
                      font-size:10pt"></span>
                    <div align="left"><span style="font-size:11pt;
                        font-family:Calibri,Helvetica,sans-serif"></span></div>
                    <span style="font-family:Calibri,Helvetica,sans-serif;
                      font-size:10pt"></span>
                    <p align="left"><span style="font-size:10pt;
                        font-family:Calibri,Helvetica,sans-serif">Abu
                        Naser</span><br>
                    </p>
                  </div>
                </div>
              </div>
              <hr style="display:inline-block; width:98%" tabindex="-1">
              <div id="x_divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> Min Si
                  <a class="x_moz-txt-link-rfc2396E OWAAutoLink" href="mailto:msi@anl.gov" id="LPlnk191149" previewremoved="true" moz-do-not-send="true">
                    <msi@anl.gov></a><br>
                  <b>Sent:</b> Tuesday, June 26, 2018 12:54:29 PM<br>
                  <b>To:</b> <a class="x_moz-txt-link-abbreviated
                    OWAAutoLink" href="mailto:discuss@mpich.org" id="LPlnk414203" previewremoved="true" moz-do-not-send="true">
                    discuss@mpich.org</a><br>
                  <b>Subject:</b> Re: [mpich-discuss] osu_latency test:
                  why 8KB takes less time than 4KB and 2KB takes less
                  time than 1KB?</font>
                <div> </div>
              </div>
              <meta content="text/html; charset=Windows-1252">
              <div style="background-color:#FFFFFF">Hi Abu,<br>
                <br>
                I think the results are stable enough. Perhaps you could
                also try the following tests, and see if similar trend
                exists:<br>
                - MPICH/socket (set `--with-device=ch3:sock` at
                configure)<br>
                - A socket-based pingpong test without MPI. <br>
                <br>
                At this point, I could not think of any MPI-specific
                design for 2k/8k messages. My guess is that it is
                related to your network connection.
                <br>
                <br>
                Min<br>
                <br>
                <div class="x_x_moz-cite-prefix">On 2018/06/24 11:09,
                  Abu Naser wrote:<br>
                </div>
                <blockquote type="cite">
                  <meta content="text/html; charset=Windows-1252">
                  <div id="x_x_divtagdefaultwrapper" dir="ltr">
                    <div id="x_x_divtagdefaultwrapper" dir="ltr">
                      <p>Hello Min and Jeff,</p>
                      <p><br>
                      </p>
                      <p>Here is my experiment results. Default number
                        of iterations in osu_latency for 0B – 8KB is
                        10,000. With that setting I had run the
                        osu_latency 100 times and found standard
                        deviation 33 for 8KB message size.</p>
                      <p><br>
                      </p>
                      <p>So later I have set the iteration to 50,000 and
                        100,000 for 1KB – 16KB message size. Then run
                        osu_latency for 100 times for each setting and
                        take the average and standard deviation.</p>
                      <p><br>
                      </p>
                      <table width="665">
                        <colgroup><col width="99"><col width="112"><col width="118"><col width="154"><col width="140"></colgroup>
                        <tbody>
                          <tr>
                            <td width="99">
                              <p><b>Msg Size in Bytes</b></p>
                            </td>
                            <td width="112">
                              <p><b>Avg time in us (50K iterations)</b></p>
                            </td>
                            <td width="118">
                              <p><b>Avg time in us (100k iterations)</b></p>
                            </td>
                            <td width="154">
                              <p><b>Standard deviation (50K iterations)</b></p>
                            </td>
                            <td width="140">
                              <p><b>Standard deviation (100K iterations)</b></p>
                            </td>
                          </tr>
                          <tr>
                            <td width="99">
                              <p>1k</p>
                            </td>
                            <td width="112">
                              <p>85.10</p>
                            </td>
                            <td width="118">
                              <p>84.9</p>
                            </td>
                            <td width="154">
                              <p>0.55</p>
                            </td>
                            <td width="140">
                              <p>0.45</p>
                            </td>
                          </tr>
                          <tr>
                            <td width="99">
                              <p>2k</p>
                            </td>
                            <td width="112">
                              <p>75.79</p>
                            </td>
                            <td width="118">
                              <p>74.63</p>
                            </td>
                            <td width="154">
                              <p>5.09</p>
                            </td>
                            <td width="140">
                              <p>4.44</p>
                            </td>
                          </tr>
                          <tr>
                            <td width="99">
                              <p>4k</p>
                            </td>
                            <td width="112">
                              <p>273.80</p>
                            </td>
                            <td width="118">
                              <p>274.71</p>
                            </td>
                            <td width="154">
                              <p>4.18</p>
                            </td>
                            <td width="140">
                              <p>2.45</p>
                            </td>
                          </tr>
                          <tr>
                            <td width="99">
                              <p>8k</p>
                            </td>
                            <td width="112">
                              <p>258.56</p>
                            </td>
                            <td width="118">
                              <p>249.83</p>
                            </td>
                            <td width="154">
                              <p>21.14</p>
                            </td>
                            <td width="140">
                              <p>28</p>
                            </td>
                          </tr>
                          <tr>
                            <td height="24" width="99">
                              <p>16k</p>
                            </td>
                            <td width="112">
                              <p>281.31</p>
                            </td>
                            <td width="118">
                              <p>281.02</p>
                            </td>
                            <td width="154">
                              <p>3.22</p>
                            </td>
                            <td width="140">
                              <p>4.10</p>
                            </td>
                          </tr>
                        </tbody>
                      </table>
                      <p><br>
                      </p>
                      <p><br>
                      </p>
                      <p>The standard deviation of 8K message is so high
                        and that implies it actually not producing any
                        consistent latency time. Looks like that's the
                        reason for 8K is taking less time than 4K.</p>
                      <p><br>
                      </p>
                      <p>Meanwhile, 2K has standard deviation less than
                        5 but 1K message latency timing are more densely
                        populated than 2K. So probably this is the
                        explanation for 2K message less latency time.</p>
                      <p><br>
                      </p>
                      <p>Thank you for your suggestions.</p>
                      <br>
                      <p><br>
                      </p>
                      <div id="x_x_Signature">
                        <div id="x_x_divtagdefaultwrapper" dir="ltr">
                          <p><br>
                          </p>
                          <p><span>Best Regards,</span></p>
                          <span></span>
                          <div><span></span></div>
                          <span></span>
                          <p><span>Abu Naser</span><br>
                          </p>
                        </div>
                      </div>
                    </div>
                    <hr tabindex="-1">
                    <div id="x_x_divRplyFwdMsg" dir="ltr"><b>From:</b>
                      Abu Naser<br>
                      <b>Sent:</b> Wednesday, June 20, 2018 1:48:53 PM<br>
                      <b>To:</b> <a class="x_x_moz-txt-link-abbreviated
                        x_OWAAutoLink" href="mailto:discuss@mpich.org" id="LPlnk729146" previewremoved="true" moz-do-not-send="true">
                        discuss@mpich.org</a><br>
                      <b>Subject:</b> Re: [mpich-discuss] osu_latency
                      test: why 8KB takes less time than 4KB and 2KB
                      takes less time than 1KB?
                      <div> </div>
                    </div>
                    <meta content="text/html; charset=iso-8859-1">
                    <div dir="ltr">
                      <div id="x_x_x_divtagdefaultwrapper" dir="ltr">
                        <div id="x_x_x_divtagdefaultwrapper" dir="ltr">
                          <p>Hello Min,</p>
                          <p><br>
                          </p>
                          <p>Thanks for the clarification.  I will do
                            the experiment.<br>
                          </p>
                          <p><br>
                          </p>
                          <div id="x_x_x_Signature">
                            <div id="x_x_x_divtagdefaultwrapper" dir="ltr">
                              <p>Thanks.</p>
                              <p><span>Best Regards,</span></p>
                              <span></span>
                              <div><span></span></div>
                              <span></span>
                              <p><span>Abu Naser</span><br>
                              </p>
                            </div>
                          </div>
                        </div>
                        <hr tabindex="-1">
                        <div id="x_x_x_divRplyFwdMsg" dir="ltr"><b>From:</b>
                          Min Si <a class="x_x_moz-txt-link-rfc2396E
                            x_OWAAutoLink" href="mailto:msi@anl.gov" id="LPlnk558260" previewremoved="true" moz-do-not-send="true">
                            <msi@anl.gov></a><br>
                          <b>Sent:</b> Wednesday, June 20, 2018 1:39:30
                          PM<br>
                          <b>To:</b> <a class="x_x_moz-txt-link-abbreviated
                            x_OWAAutoLink" href="mailto:discuss@mpich.org" id="LPlnk472728" previewremoved="true" moz-do-not-send="true">
                            discuss@mpich.org</a><br>
                          <b>Subject:</b> Re: [mpich-discuss]
                          osu_latency test: why 8KB takes less time than
                          4KB and 2KB takes less time than 1KB?
                          <div> </div>
                        </div>
                        <meta content="text/html; charset=Windows-1252">
                        <div>Hi Abu,<br>
                          <br>
                          I think Jeff means that you should run your
                          experiment with more iterations in order to
                          get a stable results.<br>
                          - Increase the iteration of for loop in each
                          execution (I think osu benchmark allows you to
                          set it)<br>
                          - Run the experiments 10 or 100 times, and
                          take the average and standard deviation.<br>
                          <br>
                          If you see a very small standard deviation
                          (e.g., <=5%), then the trend is stable and
                          you might not see such gaps.<br>
                          <br>
                          Best regards,<br>
                          Min<br>
                          <div class="x_x_x_x_moz-cite-prefix">On
                            2018/06/20 12:14, Abu Naser wrote:<br>
                          </div>
                          <blockquote type="cite">
                            <div id="x_x_x_x_divtagdefaultwrapper" dir="ltr">
                              <p>Hello Jeff,</p>
                              <p><br>
                              </p>
                              <p>Yes, I am using a switch and other
                                machines are also connected with that
                                switch.
                                <br>
                              </p>
                              <p>If I remove other machines and just use
                                my two node with the switch, then will
                                it improve the performance by 200 ~ 400
                                iterations?</p>
                              <p>Meanwhile I will give a try with a
                                single dedicated cable. <span></span><br>
                              </p>
                              <p><br>
                              </p>
                              <p>Thank you.<br>
                              </p>
                              <div id="x_x_x_x_Signature">
                                <div id="x_x_x_x_divtagdefaultwrapper" dir="ltr">
                                  <p><br>
                                  </p>
                                  <p><span>Best Regards,</span></p>
                                  <span></span>
                                  <div><span></span></div>
                                  <span></span>
                                  <p><span>Abu Naser</span><br>
                                  </p>
                                </div>
                              </div>
                            </div>
                            <hr tabindex="-1">
                            <div id="x_x_x_x_divRplyFwdMsg" dir="ltr"><b>From:</b>
                              Jeff Hammond <a class="x_x_x_x_moz-txt-link-rfc2396E
                                x_x_x_OWAAutoLink" href="mailto:jeff.science@gmail.com" id="LPlnk983157" previewremoved="true" moz-do-not-send="true">
                                <jeff.science@gmail.com></a><br>
                              <b>Sent:</b> Wednesday, June 20, 2018
                              12:52:06 PM<br>
                              <b>To:</b> MPICH<br>
                              <b>Subject:</b> Re: [mpich-discuss]
                              osu_latency test: why 8KB takes less time
                              than 4KB and 2KB takes less time than 1KB?
                              <div> </div>
                            </div>
                            <meta content="text/html; charset=utf-8">
                            <div>
                              <div dir="ltr">Is the ethernet connection
                                a single dedicated cable between the two
                                machines or are you running through a
                                switch that handles other traffic?
                                <div><br>
                                </div>
                                <div>My best guess is that this is noise
                                  and that you may be able to avoid it
                                  by running a very long time, e.g.
                                  10000 iterations.</div>
                                <div><br>
                                </div>
                                <div>Jeff</div>
                              </div>
                              <div class="x_x_x_x_x_gmail_extra"><br>
                                <div class="x_x_x_x_x_gmail_quote">On
                                  Wed, Jun 20, 2018 at 6:53 AM, Abu
                                  Naser <span dir="ltr">
                                    <<a href="mailto:an16e@my.fsu.edu" target="_blank" id="LPlnk305789" class="x_x_x_OWAAutoLink" previewremoved="true" moz-do-not-send="true">an16e@my.fsu.edu</a>></span>
                                  wrote:<br>
                                  <blockquote class="x_x_x_x_x_gmail_quote">
                                    <div dir="ltr">
                                      <div id="x_x_x_x_x_m_6077755676379859201divtagdefaultwrapper" dir="ltr">
                                        <p><br>
                                        </p>
                                        <p>Good day to all,</p>
                                        <p><br>
                                        </p>
                                        <p>I had run point to point
                                          osu_latency test in two nodes
                                          for 200 times.  Followings are
                                          the average time in
                                          microsecond for various size
                                          of the messages -</p>
                                        <div>1KB    84.8514 us<br>
                                          <span>2KB    73.52535</span>
                                          us<br>
                                          4KB    272.55275 us<br>
                                          <span>8KB    234.86385</span>
                                          us<br>
                                          16KB    288.88 us<br>
                                          32KB    523.3725 us<br>
                                          64KB    910.4025 us</div>
                                        <p><br>
                                        </p>
                                        <p>From the above looks like,
                                          2KB message has less latency
                                          than 1 KB and 8KB has less
                                          latency than 4KB.
                                          <br>
                                        </p>
                                        <p>I was looking for explanation
                                          of this behavior  but did not
                                          get any.</p>
                                        <p><br>
                                        </p>
                                        <ol>
                                          <li><span>MPIR_CVAR_CH3_EAGER_MAX_MSG_<wbr>SIZE</span><span>
                                              is set to 128KB. So none
                                              of the above message size
                                              is using Rendezvous
                                              protocol. Is there any
                                              partition inside eager
                                              protocol (e.g. 0 - 512
                                              bytes, 1KB - 8KB, 16KB -
                                              64KB)? If yes then what
                                              are the boundaries for
                                              them? Can I log them with
                                              debug-event-logging? </span><br>
                                          </li>
                                        </ol>
                                        <p><br>
                                        </p>
                                        <p>Setup I am using:</p>
                                        <p>- two nodes has intel core
                                          i7, one with 16gb memory
                                          another one 8gb</p>
                                        <p>- mpich 3.2.1, configured and
                                          build to use nemesis tcp</p>
                                        <p>- 1gb Ethernet connection</p>
                                        <p>- NFS is using for sharing<br>
                                        </p>
                                        <p>- osu_latency : uses MPI_Send
                                          and MPI_Recv</p>
                                        <p>- <span>MPIR_CVAR_CH3_EAGER_MAX_MSG_<wbr>SIZE</span>=
                                          <span>131072</span> (128KB)<br>
                                        </p>
                                        <p><br>
                                        </p>
                                        <p>Can anyone help me on that?
                                          Thanks in advance.<br>
                                        </p>
                                        <p><br>
                                        </p>
                                        <p><br>
                                        </p>
                                        <div id="x_x_x_x_x_m_6077755676379859201Signature">
                                          <div id="x_x_x_x_x_m_6077755676379859201divtagdefaultwrapper" dir="ltr">
                                            <p><br>
                                            </p>
                                            <p><span>Best Regards,</span></p>
                                            <span></span>
                                            <div><span></span></div>
                                            <span></span>
                                            <p><span>Abu Naser</span><br>
                                            </p>
                                          </div>
                                        </div>
                                      </div>
                                    </div>
                                    <br>
                                    ______________________________<wbr>_________________<br>
                                    discuss mailing list     <a href="mailto:discuss@mpich.org" id="LPlnk816471" class="x_x_x_OWAAutoLink" previewremoved="true" moz-do-not-send="true">discuss@mpich.org</a><br>
                                    To manage subscription options or
                                    unsubscribe:<br>
                                    <a href="https://lists.mpich.org/mailman/listinfo/discuss" rel="noreferrer" target="_blank" id="LPlnk624595" class="x_x_x_OWAAutoLink" previewremoved="true" moz-do-not-send="true">https://lists.mpich.org/<wbr>mailman/listinfo/discuss</a><br>
                                    <br>
                                  </blockquote>
                                </div>
                                <br>
                                <br>
                                <div><br>
                                </div>
                                -- <br>
                                <div class="x_x_x_x_x_gmail_signature">Jeff
                                  Hammond<br>
                                  <a href="mailto:jeff.science@gmail.com" target="_blank" id="LPlnk314993" class="x_x_x_OWAAutoLink" previewremoved="true" moz-do-not-send="true">jeff.science@gmail.com</a><br>
                                  <a href="http://jeffhammond.github.io/" target="_blank" id="LPlnk861434" class="x_x_x_OWAAutoLink" previewremoved="true" moz-do-not-send="true">http://jeffhammond.github.io/</a></div>
                              </div>
                            </div>
                            <br>
                            <fieldset class="x_x_x_x_mimeAttachmentHeader"></fieldset>
                            <br>
                            <pre>_______________________________________________
discuss mailing list     <a class="x_x_x_x_moz-txt-link-abbreviated x_x_x_OWAAutoLink" href="mailto:discuss@mpich.org" id="LPlnk657371" previewremoved="true" moz-do-not-send="true">discuss@mpich.org</a>
To manage subscription options or unsubscribe:
<a class="x_x_x_x_moz-txt-link-freetext x_x_x_OWAAutoLink" href="https://lists.mpich.org/mailman/listinfo/discuss" id="LPlnk669988" previewremoved="true" moz-do-not-send="true">https://lists.mpich.org/mailman/listinfo/discuss</a>
</pre>
                          </blockquote>
                          <br>
                        </div>
                      </div>
                    </div>
                  </div>
                  <br>
                  <fieldset class="x_x_mimeAttachmentHeader"></fieldset>
                  <br>
                  <pre>_______________________________________________
discuss mailing list     <a class="x_x_moz-txt-link-abbreviated x_OWAAutoLink" href="mailto:discuss@mpich.org" id="LPlnk832953" previewremoved="true" moz-do-not-send="true">discuss@mpich.org</a>
To manage subscription options or unsubscribe:
<a class="x_x_moz-txt-link-freetext x_OWAAutoLink" href="https://lists.mpich.org/mailman/listinfo/discuss" id="LPlnk481779" previewremoved="true" moz-do-not-send="true">https://lists.mpich.org/mailman/listinfo/discuss</a>
</pre>
                </blockquote>
                <br>
              </div>
            </div>
            <br>
            <fieldset class="x_mimeAttachmentHeader"></fieldset>
            <br>
            <pre>_______________________________________________
discuss mailing list     <a class="x_moz-txt-link-abbreviated OWAAutoLink" href="mailto:discuss@mpich.org" id="LPlnk408695" previewremoved="true" moz-do-not-send="true">discuss@mpich.org</a>
To manage subscription options or unsubscribe:
<a class="x_moz-txt-link-freetext OWAAutoLink" href="https://lists.mpich.org/mailman/listinfo/discuss" id="LPlnk572504" previewremoved="true" moz-do-not-send="true">https://lists.mpich.org/mailman/listinfo/discuss</a>
</pre>
          </blockquote>
          <br>
        </div>
      </div>
      <!--'"--><br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <br>
      <pre wrap="">_______________________________________________
discuss mailing list     <a class="moz-txt-link-abbreviated" href="mailto:discuss@mpich.org">discuss@mpich.org</a>
To manage subscription options or unsubscribe:
<a class="moz-txt-link-freetext" href="https://lists.mpich.org/mailman/listinfo/discuss">https://lists.mpich.org/mailman/listinfo/discuss</a>
</pre>
    </blockquote>
    <br>
  </body>
</html>