<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"><style>body { line-height: 1.5; }blockquote { margin-top: 0px; margin-bottom: 0px; margin-left: 0.5em; }body { font-size: 10.5pt; font-family: 微软雅黑; color: rgb(0, 0, 0); line-height: 1.5; }</style></head><body>
<div><span></span>Hi Rob, </div><div> Can we increase the number of open files on Mira to 1 million? <span style="font-size: 10.5pt; line-height: 1.5; background-color: window;">Currently the number of open files is set to be 65536, and users can increase it manually to its hard limitation of 81920. </span></div><div> Can we increase this hard limitation to 1 million for Mira for a one week test on MPI IO in Mira? Thanks. </div><div><br></div><div>---------------<span style="color: rgb(0, 0, 0); background-color: rgba(0, 0, 0, 0);"> /etc/security/limits.conf</span>---------------------------------</div><div><span style="color: rgb(0, 0, 0); background-color: rgba(0, 0, 0, 0);">root soft nofile 65536<br>root hard nofile 81920 <--</span></div>
<div><br></div><div>Best, </div><hr style="width: 210px; height: 1px;" color="#b5c4df" size="1" align="left">
<div><span><div style="MARGIN: 10px; FONT-FAMILY: verdana; FONT-SIZE: 10pt"><div>jt.meng@siat.ac.cn</div></div></span></div>
<blockquote style="margin-top: 0px; margin-bottom: 0px; margin-left: 0.5em;"><div> </div><div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm"><div style="PADDING-RIGHT: 8px; PADDING-LEFT: 8px; FONT-SIZE: 12px;FONT-FAMILY:tahoma;COLOR:#000000; BACKGROUND: #efefef; PADDING-BOTTOM: 8px; PADDING-TOP: 8px"><div><b>From:</b> <a href="mailto:robl@mcs.anl.gov">Latham, Robert J.</a></div><div><b>Date:</b> 2017-04-19 22:35</div><div><b>To:</b> <a href="mailto:discuss@mpich.org">discuss@mpich.org</a></div><div><b>Subject:</b> Re: [mpich-discuss] Installation on Blue Gene/Q</div></div></div><div><div>On Wed, 2017-04-19 at 13:58 +0200, pramod kumbhar wrote:</div>
<div>> Hi Rob,</div>
<div>> </div>
<div>> Is this already installed/avaialble on MIRA? I would like to try this</div>
<div>> to check some mpi i/o hints.</div>
<div> </div>
<div>Yes, there are several versions available:</div>
<div> </div>
<div>manual progress optimized:</div>
<div>/projects/aurora_app/mpich3-ch4-ofi/install/gnu/bin</div>
<div> </div>
<div>auto progress optimized:</div>
<div>/projects/aurora_app/mpich3-ch4-ofi/install-auto/gnu/bin</div>
<div> </div>
<div>manual progress debug:</div>
<div>/projects/aurora_app/mpich3-ch4-ofi/install/gnu.debug/bin</div>
<div> </div>
<div>auto progress debug:</div>
<div>/projects/aurora_app/mpich3-ch4-ofi/install-auto/gnu.debug/bin</div>
<div> </div>
<div>There are some performance considerations (both positive and negative)</div>
<div>to consider:</div>
<div> </div>
<div>Current performance improvements versus [IBM's vendor-supplied MPICH]:</div>
<div> </div>
<div>1.) Pt2pt latency - better than 2x speedup for small messages, large</div>
<div>ones with few ranks-per-node less so due to detriment #2 listed below.</div>
<div> </div>
<div>2.) RMA latency - exponentially better for small messages, then becomes</div>
<div>gradually even for large messages in the case of put and get.</div>
<div> </div>
<div>Current performance detriments versus [IBM's vendor-supplied MPICH]:</div>
<div> </div>
<div>1.) No optimized collectives -- all collectives are pt2pt algorithms in</div>
<div>MPICH CH4. There is a plan to develop optimized collectives via OFI</div>
<div>triggered operations but that work wouldn't begin until the April-May</div>
<div>timeframe at the earliest. There is also a plan to make optimized</div>
<div>pt2pt collectives available sooner.</div>
<div> </div>
<div>2.) Only 1 rget injection fifo utilized per rank to pt2pt communication</div>
<div>--- pami can utilize up all 11 to maximize internode bandwidth, you</div>
<div>would mainly see this have an impact if you are running < 8 ranks per</div>
<div>node and passing large messages.</div>
<div> </div>
<div>==rob</div>
<div>_______________________________________________</div>
<div>discuss mailing list discuss@mpich.org</div>
<div>To manage subscription options or unsubscribe:</div>
<div>https://lists.mpich.org/mailman/listinfo/discuss</div>
</div></blockquote>
</body></html>