<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
p.xmsonormal, li.xmsonormal, div.xmsonormal
{mso-style-name:x_msonormal;
margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.xmsolistparagraph, li.xmsolistparagraph, div.xmsolistparagraph
{mso-style-name:x_msolistparagraph;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle23
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1526210262;
mso-list-template-ids:2052651682;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Thanks Hui, is the spawned process on the local host, or the remote host or both?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Kurt<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Zhou, Hui <zhouh@anl.gov> <br>
<b>Sent:</b> Friday, April 1, 2022 4:20 PM<br>
<b>To:</b> discuss@mpich.org<br>
<b>Cc:</b> Mccall, Kurt E. (MSFC-EV41) <kurt.e.mccall@nasa.gov><br>
<b>Subject:</b> [EXTERNAL] Re: Hydra WARNING: too many ssh connections<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">Every time you call MPI_Comm_spawn, hydra will launch a ssh (for each host) to create a proxy. It is certainly not ideal for applications relying on spawning many processes.<o:p></o:p></span></p>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="98%" align="center">
</div>
<div id="divRplyFwdMsg">
<p class="MsoNormal"><b><span style="color:black">From:</span></b><span style="color:black"> Mccall, Kurt E. (MSFC-EV41) via discuss <<a href="mailto:discuss@mpich.org">discuss@mpich.org</a>><br>
<b>Sent:</b> Friday, April 1, 2022 4:08 PM<br>
<b>To:</b> <a href="mailto:discuss@mpich.org">discuss@mpich.org</a> <<a href="mailto:discuss@mpich.org">discuss@mpich.org</a>><br>
<b>Cc:</b> Mccall, Kurt E. (MSFC-EV41) <<a href="mailto:kurt.e.mccall@nasa.gov">kurt.e.mccall@nasa.gov</a>><br>
<b>Subject:</b> [mpich-discuss] Hydra WARNING: too many ssh connections</span> <o:p>
</o:p></p>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="xmsonormal">Hi, you provided the following information about the warning “too many ssh connections”:<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:10.0pt;font-family:"Courier New"">The particular warning is issued by hydra, MPICH’s process manager. Following excerpt is the comment in that source code:</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:10.0pt;font-family:"Courier New""> </span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:10.0pt;font-family:"Courier New""> /* ssh has many types of security controls that do not allow a</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:10.0pt;font-family:"Courier New""> * user to ssh to the same node multiple times very</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:10.0pt;font-family:"Courier New""> * quickly. If this happens, the ssh daemons disables ssh</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:10.0pt;font-family:"Courier New""> * connections causing the job to fail. This is basically a</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:10.0pt;font-family:"Courier New""> * hack to slow down ssh connections to the same node. We</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:10.0pt;font-family:"Courier New""> * check for offset == 0 before applying this hack, so we only</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:10.0pt;font-family:"Courier New""> * slow down the cases where ssh is being used, and not the</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:10.0pt;font-family:"Courier New""> * cases where we fall back to fork. */</span><o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">Is this just during an initial ssh connection attempt? I’m trying to figure out where my code is triggering this warning. Could it be from<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<ol style="margin-top:0in" start="1" type="1">
<li class="xmsolistparagraph" style="margin-left:0in;mso-list:l0 level1 lfo1">MPI_Intercomm_create<o:p></o:p></li><li class="xmsolistparagraph" style="margin-left:0in;mso-list:l0 level1 lfo1">MPI_Comm_spawn<o:p></o:p></li><li class="xmsolistparagraph" style="margin-left:0in;mso-list:l0 level1 lfo1">others?<o:p></o:p></li></ol>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">I’m calling mpiexec with the “—launcher ssh” option, MPICH 4.0.1.<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">Thanks,<o:p></o:p></p>
<p class="xmsonormal">Kurt<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
</div>
</div>
</div>
</body>
</html>