<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">Hello,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">We used to use mpich2-1.2 with gforker process manager (Intel 10.1 compiler).<o:p></o:p></p>
<p class="MsoNormal">We are transitioning mpich-3.2 with gforker process manager (Intel 13.1 compiler) (configuration options are the same as before:
<span style="font-size:10.0pt;font-family:Consolas">./configure --prefix=$MPI_DIR --enable-g=debug,meminit --enable-fast=O2 --with-pm=gforker --enable-cxx --enable-fortran CC=icc CXX=icpc FC=ifort F77=ifort)<br>
<br>
</span><o:p></o:p></p>
<p class="MsoNormal">We have noticed extreme slowdowns with the new installation of mpich on the same machine (shared memory Linux server with CentOS 7) when attempting to use hyperthreading. The old executable works just fine on the same machine, but the new
executable using mpich-3.2 is running 10-20x slower when going past the number of physical cores on the machine. We believe we have traced it to our MPI_AllReduce functions. This is a Fortran code.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">In the meantime I’m going to try to install the old mpich2-1.2 again (we had a machine OS upgrade) and see what happens.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Any ideas?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thank you,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Emily Shemon, Ph.D. <o:p></o:p></p>
<p class="MsoNormal">Argonne National Laboratory<o:p></o:p></p>
<p class="MsoNormal">Nuclear Engineering Division & <o:p></o:p></p>
<p class="MsoNormal">Argonne Leadership Computing Facility<o:p></o:p></p>
<p class="MsoNormal">P: 630-252-4477<o:p></o:p></p>
<p class="MsoNormal">E: <a href="mailto:eshemon@anl.gov">eshemon@anl.gov</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>