No subject


Tue Jun 18 13:52:11 CDT 2019 

e algorithm. i.e. the problem may not lie with the inefficiency of your com=
munication between threads but just that your algorithm is not keeping the =
processor busy enough for a large number of threads. The only way that you =
will know for sure whether this is a comms issue or an algorithmic one is t=
o use a profiling tool, such as Vampir or Paraver.

With the profiling result you will be able to determine whether you need to=
 make algorithmic changes in your bulk processing and enhance comms as per =
Huiwei's notes.

I would be interested to know what your profiling shows.

Regards, bob

On Thu, Oct 23, 2014 at 12:02 AM, Qiguo Jing <qjing at trinityconsultants.com<=
mailto:qjing at trinityconsultants.com>> wrote:
Hi All,

We have a parallel program running on a cluster.  We recently found a case,=
 which decreases the CPU usage and increase the run-time when increases Nod=
es.   Below is the results table.

The particular run requires a lot of data communication between nodes.

Any thoughts about this phenomena?  Or is there any way we can improve the =
CPU usage when using higher number of nodes?

Average CPU Usage (%)

Number of Nodes

Number of Threads/Node

100

1

8

92

2

8

50

3

8

40

4

8

35

5

8

30

6

8

25

7

8

20

8

8

20

8

4



Thanks!

_________________________________________________________________________

The information transmitted is intended only for the person or entity to
which it is addressed and may contain confidential and/or privileged
material. Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by persons or
entities other than the intended recipient is prohibited. If you received
this in error, please contact the sender and delete the material from any
computer.
_________________________________________________________________________

_______________________________________________
discuss mailing list     discuss at mpich.org<mailto:discuss at mpich.org>
To manage subscription options or unsubscribe:
https://lists.mpich.org/mailman/listinfo/discuss


_________________________________________________________________________

The information transmitted is intended only for the person or entity to
which it is addressed and may contain confidential and/or privileged
material. Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by persons or
entities other than the intended recipient is prohibited. If you received
this in error, please contact the sender and delete the material from any
computer.
_________________________________________________________________________

_________________________________________________________________________

The information transmitted is intended only for the person or entity to
which it is addressed and may contain confidential and/or privileged
material. Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by persons or
entities other than the intended recipient is prohibited. If you received
this in error, please contact the sender and delete the material from any
computer.
_________________________________________________________________________

_______________________________________________
discuss mailing list     discuss at mpich.org<mailto:discuss at mpich.org>
To manage subscription options or unsubscribe:
https://lists.mpich.org/mailman/listinfo/discuss



--
/ Ruben FAELENS
+32 494 06 72 59

--=20
_________________________________________________________________________

The information transmitted is intended only for the person or entity to
which it is addressed and may contain confidential and/or privileged
material. Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by persons or
entities other than the intended recipient is prohibited. If you received
this in error, please contact the sender and delete the material from any
computer.
_________________________________________________________________________

--_000_9D079B25EA4E3149B260846F07AE80754C5944A3TCIEXCH03ustrin_
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:=
//www.w3.org/TR/REC-html40"><head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dutf-8">
<meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
	{font-family:=E5=AE=8B=E4=BD=93;
	panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
	{font-family:"Cambria Math";
	panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
	{font-family:"\@=E5=AE=8B=E4=BD=93";
	panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0in;
	margin-bottom:.0001pt;
	font-size:12.0pt;
	font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
	{mso-style-priority:99;
	color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{mso-style-priority:99;
	color:purple;
	text-decoration:underline;}
span.EmailStyle17
	{mso-style-type:personal-reply;
	font-family:"Calibri",sans-serif;
	color:#1F497D;}
.MsoChpDefault
	{mso-style-type:export-only;
	font-family:"Calibri",sans-serif;}
@page WordSection1
	{size:8.5in 11.0in;
	margin:1.0in 1.25in 1.0in 1.25in;}
div.WordSection1
	{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=3D"EN-US" link=3D"blue" vlink=3D"purple">
<div class=3D"WordSection1">
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca=
libri",sans-serif;color:#1F497D">Hi Ruben,<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca=
libri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca=
libri",sans-serif;color:#1F497D">You are right! My algorithm does what=
 you described.
<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca=
libri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca=
libri",sans-serif;color:#1F497D">I will record the timestamp for each =
thread and every event.  Thanks for your suggestions! 
<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca=
libri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca=
libri",sans-serif;color:#1F497D">Qiguo<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca=
libri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class=3D"MsoNormal"><b><span style=3D"font-size:11.0pt;font-family:&quot=
;Calibri",sans-serif">From:</span></b><span style=3D"font-size:11.0pt;=
font-family:"Calibri",sans-serif"> parasietje at gmail.com [mailto:p=
arasietje at gmail.com]
<b>On Behalf Of </b>Ruben Faelens<br>
<b>Sent:</b> Thursday, October 23, 2014 10:58 AM<br>
<b>To:</b> discuss at mpich.org<br>
<b>Subject:</b> Re: [mpich-discuss] CPU usage versus Nodes, Threads<o:p></o=
:p></span></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<div>
<p class=3D"MsoNormal">Hi Qiguo,<o:p></o:p></p>
<div>
<p class=3D"MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class=3D"MsoNormal">You should try to collect performance statistics, es=
pecially regarding your specific nodes and what they are doing at every mom=
ent in time.<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal">If I understand correctly, your algorithm does the f=
ollowing:<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal">- Master thread: read in data, split it up into piec=
es, transfer pieces to slaves<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal">- Slave thread: do calculation, transfer data back t=
o master<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal">- Master: recombine data, do calculation, split data=
 back up, transfer pieces to slaves<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal">- etc...<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class=3D"MsoNormal">The reason you do not see linear performance scaling=
 could be due to the following:<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal">- The master thread recombining and splitting the da=
ta set may be responsible for a large part of the work (and therefore is th=
e bottleneck)<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal">- Work is not divided equally. A significant part of=
 the time is spent waiting on one slave node who has a more difficult probl=
em (takes longer) than the rest.<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal">- There is a common dataset. I/O takes a larger part=
 of the time when more slave nodes are used.<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class=3D"MsoNormal">The only way to know for sure, is to simply generate=
 a log file that shows the time every process starts and ends a specific pr=
ocess step. Output the time when <o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal">- the slave starts receiving data<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal">- starts calculation<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal">- starts sending results back<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal">- starts waiting for his next piece of data<o:p></o:=
p></p>
</div>
<div>
<p class=3D"MsoNormal">This will clearly show you what each node is doing a=
t each moment in time, and should identify the bottleneck.<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class=3D"MsoNormal">/ Ruben<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal"><o:p> </o:p></p>
</div>
</div>
<div>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<div>
<p class=3D"MsoNormal">On Thu, Oct 23, 2014 at 5:27 PM, Qiguo Jing <<a h=
ref=3D"mailto:qjing at trinityconsultants.com" target=3D"_blank">qjing at trinity=
consultants.com</a>> wrote:<o:p></o:p></p>
<blockquote style=3D"border:none;border-left:solid #CCCCCC 1.0pt;padding:0i=
n 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"><span style=3D"font-size:11.0pt;font-family:"Calibri",sa=
ns-serif;color:#1F497D">Hi Bob,</span><o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"><span style=3D"font-size:11.0pt;font-family:"Calibri",sa=
ns-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"><span style=3D"font-size:11.0pt;font-family:"Calibri",sa=
ns-serif;color:#1F497D">Thanks for your suggestions.   Here are m=
ore tests.  We actually have three clusters.</span><o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"><span style=3D"font-size:11.0pt;font-family:"Calibri",sa=
ns-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"><span style=3D"font-size:11.0pt;font-family:"Calibri",sa=
ns-serif;color:#1F497D">Cluster 1 and 2:  8 nodes, (2 Processors, 4 co=
res/processor, no HT =E2=80=93 Total 8 Threads)/node</span><o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"><span style=3D"font-size:11.0pt;font-family:"Calibri",sa=
ns-serif;color:#1F497D">Cluster 3:      &nbsp=
;       8 nodes, (1 Processors, 4 cores /proc=
essor, HT =E2=80=93 Total 8 Threads )/node</span><o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"><span style=3D"font-size:11.0pt;font-family:"Calibri",sa=
ns-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"><span style=3D"font-size:11.0pt;font-family:"Calibri",sa=
ns-serif;color:#1F497D">We also have a standalone machine:  2 processo=
rs, 6 cores/processor, HT =E2=80=93 total 24 threads.</span><o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"><span style=3D"font-size:11.0pt;font-family:"Calibri",sa=
ns-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"><span style=3D"font-size:11.0pt;font-family:"Calibri",sa=
ns-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"><span style=3D"font-size:11.0pt;font-family:"Calibri",sa=
ns-serif;color:#1F497D">For one particular case: 
</span><o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"><span style=3D"font-size:11.0pt;font-family:"Calibri",sa=
ns-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"><span style=3D"font-size:11.0pt;font-family:"Calibri",sa=
ns-serif;color:#1F497D">Cluster 1 and 2 take 48 min to finish with 8 nodes,=
 8 threads/node,  60% CPU usage;  53 min to finish
 with 3 nodes, 8 threads/node, 90% CPU usage;</span><o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"><span style=3D"font-size:11.0pt;font-family:"Calibri",sa=
ns-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"><span style=3D"font-size:11.0pt;font-family:"Calibri",sa=
ns-serif;color:#1F497D">Cluster 3 takes 227 min to finish with 8 nodes, 8 t=
hreads/node, 20% CPU usage; 207 min to finish with
 3 nodes, 8 threads/node, 50% CPU usage;</span><o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"><span style=3D"font-size:11.0pt;font-family:"Calibri",sa=
ns-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"><span style=3D"font-size:11.0pt;font-family:"Calibri",sa=
ns-serif;color:#1F497D">Standalone machine takes 82 min to finish with 24 t=
hreads, 100% CPU usage.</span><o:p></o:p></p>
<div>
<div>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"><span style=3D"font-size:11.0pt;font-family:"Calibri",sa=
ns-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"><span style=3D"font-size:11.0pt;font-family:"Calibri",sa=
ns-serif;color:#1F497D">It looks like with 24 threads, they should be prett=
y busy?   Could the above phenomena be a hardware
 issue more than software?</span><o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"><span style=3D"font-size:11.0pt;font-family:"Calibri",sa=
ns-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"><span style=3D"font-size:11.0pt;font-family:"Calibri",sa=
ns-serif;color:#1F497D">Qiguo</span><o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"><span style=3D"font-size:11.0pt;font-family:"Calibri",sa=
ns-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"><b><span style=3D"font-size:11.0pt;font-family:"Calibri"=
,sans-serif">From:</span></b><span style=3D"font-size:11.0pt;font-family:&q=
uot;Calibri",sans-serif"> Bob Ilgner [<a href=3D"mailto:bobilgner at gmai=
l.com" target=3D"_blank">mailto:bobilgner at gmail.com</a>]
<br>
<b>Sent:</b> Thursday, October 23, 2014 1:11 AM<br>
<b>To:</b> <a href=3D"mailto:discuss at mpich.org" target=3D"_blank">discuss at m=
pich.org</a><br>
<b>Subject:</b> Re: [mpich-discuss] CPU usage versus Nodes, Threads</span><=
o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"> <o:p></o:p></p>
<div>
<div>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto">Hi Qiguo,<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"> <o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto">From the results table it looks as if you are using a computa=
tionally sparse algorithm. i.e. the problem may not lie with the inefficien=
cy of your communication between threads
 but just that your algorithm is not keeping the processor busy enough for =
a large number of threads. The only way that you will know for sure whether=
 this is a comms issue or an algorithmic one is to use a profiling tool, su=
ch as Vampir or Paraver.<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"> <o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto">With the profiling result you will be able to determine whether yo=
u need to make algorithmic changes in your bulk processing and enhance comm=
s as per Huiwei's notes.<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"> <o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto">I would be interested to know what your profiling shows.<o:p></o:p=
></p>
</div>
<div>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"> <o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto">Regards, bob<o:p></o:p></p>
</div>
</div>
<div>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"> <o:p></o:p></p>
<div>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto">On Thu, Oct 23, 2014 at 12:02 AM, Qiguo Jing <<a href=3D"mailto=
:qjing at trinityconsultants.com" target=3D"_blank">qjing at trinityconsultants.c=
om</a>> wrote:<o:p></o:p></p>
<blockquote style=3D"border:none;border-left:solid #CCCCCC 1.0pt;padding:0i=
n 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-=
bottom:5.0pt">
<div>
<div>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto">Hi All,<o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"> <o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto">We have a parallel program running on a cluster.  We recently=
 found a case, which decreases the CPU usage and increase the run-time when=
 increases Nodes.   Below is the results
 table.<o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"> <o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto">The particular run requires a lot of data communication between no=
des.
<o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"> <o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto">Any thoughts about this phenomena?  Or is there any way we ca=
n improve the CPU usage when using higher number of nodes?
<o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"> <o:p></o:p></p>
<table class=3D"MsoNormalTable" border=3D"0" cellspacing=3D"0" cellpadding=
=3D"0" width=3D"441" style=3D"width:331.0pt;border-collapse:collapse">
<tbody>
<tr style=3D"height:15.0pt">
<td width=3D"153" nowrap=3D"" valign=3D"bottom" style=3D"width:115.0pt;bord=
er:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:15.0pt">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">Average CPU Usage (%)</span><o:p></o:p></p>
</td>
<td width=3D"119" nowrap=3D"" valign=3D"bottom" style=3D"width:89.0pt;borde=
r:solid windowtext 1.0pt;border-left:none;padding:0in 5.4pt 0in 5.4pt;heigh=
t:15.0pt;border-color:currentColor">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">Number of Nodes</span><o:p></o:p></p>
</td>
<td width=3D"169" nowrap=3D"" valign=3D"bottom" style=3D"width:127.0pt;bord=
er:solid windowtext 1.0pt;border-left:none;padding:0in 5.4pt 0in 5.4pt;heig=
ht:15.0pt;border-color:currentColor">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">Number of Threads/Node</span><o:p></o:p></p>
</td>
</tr>
<tr style=3D"height:15.0pt">
<td width=3D"153" nowrap=3D"" valign=3D"bottom" style=3D"width:115.0pt;bord=
er:solid windowtext 1.0pt;border-top:none;padding:0in 5.4pt 0in 5.4pt;heigh=
t:15.0pt;border-color:currentColor windowtext windowtext">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">100</span><o:p></o:p></p>
</td>
<td width=3D"119" nowrap=3D"" valign=3D"bottom" style=3D"width:89.0pt;borde=
r-top:none;border-left:none;border-bottom:solid windowtext 1.0pt;border-rig=
ht:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:15.0pt;border-=
color:currentColor windowtext windowtext currentColor">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">1</span><o:p></o:p></p>
</td>
<td width=3D"169" nowrap=3D"" valign=3D"bottom" style=3D"width:127.0pt;bord=
er-top:none;border-left:none;border-bottom:solid windowtext 1.0pt;border-ri=
ght:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:15.0pt;border=
-color:currentColor windowtext windowtext currentColor">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">8</span><o:p></o:p></p>
</td>
</tr>
<tr style=3D"height:15.0pt">
<td width=3D"153" nowrap=3D"" valign=3D"bottom" style=3D"width:115.0pt;bord=
er:solid windowtext 1.0pt;border-top:none;padding:0in 5.4pt 0in 5.4pt;heigh=
t:15.0pt;border-color:currentColor windowtext windowtext">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">92</span><o:p></o:p></p>
</td>
<td width=3D"119" nowrap=3D"" valign=3D"bottom" style=3D"width:89.0pt;borde=
r-top:none;border-left:none;border-bottom:solid windowtext 1.0pt;border-rig=
ht:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:15.0pt;border-=
color:currentColor windowtext windowtext currentColor">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">2</span><o:p></o:p></p>
</td>
<td width=3D"169" nowrap=3D"" valign=3D"bottom" style=3D"width:127.0pt;bord=
er-top:none;border-left:none;border-bottom:solid windowtext 1.0pt;border-ri=
ght:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:15.0pt;border=
-color:currentColor windowtext windowtext currentColor">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">8</span><o:p></o:p></p>
</td>
</tr>
<tr style=3D"height:15.0pt">
<td width=3D"153" nowrap=3D"" valign=3D"bottom" style=3D"width:115.0pt;bord=
er:solid windowtext 1.0pt;border-top:none;padding:0in 5.4pt 0in 5.4pt;heigh=
t:15.0pt;border-color:currentColor windowtext windowtext">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">50</span><o:p></o:p></p>
</td>
<td width=3D"119" nowrap=3D"" valign=3D"bottom" style=3D"width:89.0pt;borde=
r-top:none;border-left:none;border-bottom:solid windowtext 1.0pt;border-rig=
ht:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:15.0pt;border-=
color:currentColor windowtext windowtext currentColor">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">3</span><o:p></o:p></p>
</td>
<td width=3D"169" nowrap=3D"" valign=3D"bottom" style=3D"width:127.0pt;bord=
er-top:none;border-left:none;border-bottom:solid windowtext 1.0pt;border-ri=
ght:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:15.0pt;border=
-color:currentColor windowtext windowtext currentColor">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">8</span><o:p></o:p></p>
</td>
</tr>
<tr style=3D"height:15.0pt">
<td width=3D"153" nowrap=3D"" valign=3D"bottom" style=3D"width:115.0pt;bord=
er:solid windowtext 1.0pt;border-top:none;padding:0in 5.4pt 0in 5.4pt;heigh=
t:15.0pt;border-color:currentColor windowtext windowtext">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">40</span><o:p></o:p></p>
</td>
<td width=3D"119" nowrap=3D"" valign=3D"bottom" style=3D"width:89.0pt;borde=
r-top:none;border-left:none;border-bottom:solid windowtext 1.0pt;border-rig=
ht:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:15.0pt;border-=
color:currentColor windowtext windowtext currentColor">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">4</span><o:p></o:p></p>
</td>
<td width=3D"169" nowrap=3D"" valign=3D"bottom" style=3D"width:127.0pt;bord=
er-top:none;border-left:none;border-bottom:solid windowtext 1.0pt;border-ri=
ght:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:15.0pt;border=
-color:currentColor windowtext windowtext currentColor">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">8</span><o:p></o:p></p>
</td>
</tr>
<tr style=3D"height:15.0pt">
<td width=3D"153" nowrap=3D"" valign=3D"bottom" style=3D"width:115.0pt;bord=
er:solid windowtext 1.0pt;border-top:none;padding:0in 5.4pt 0in 5.4pt;heigh=
t:15.0pt;border-color:currentColor windowtext windowtext">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">35</span><o:p></o:p></p>
</td>
<td width=3D"119" nowrap=3D"" valign=3D"bottom" style=3D"width:89.0pt;borde=
r-top:none;border-left:none;border-bottom:solid windowtext 1.0pt;border-rig=
ht:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:15.0pt;border-=
color:currentColor windowtext windowtext currentColor">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">5</span><o:p></o:p></p>
</td>
<td width=3D"169" nowrap=3D"" valign=3D"bottom" style=3D"width:127.0pt;bord=
er-top:none;border-left:none;border-bottom:solid windowtext 1.0pt;border-ri=
ght:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:15.0pt;border=
-color:currentColor windowtext windowtext currentColor">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">8</span><o:p></o:p></p>
</td>
</tr>
<tr style=3D"height:15.0pt">
<td width=3D"153" nowrap=3D"" valign=3D"bottom" style=3D"width:115.0pt;bord=
er:solid windowtext 1.0pt;border-top:none;padding:0in 5.4pt 0in 5.4pt;heigh=
t:15.0pt;border-color:currentColor windowtext windowtext">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">30</span><o:p></o:p></p>
</td>
<td width=3D"119" nowrap=3D"" valign=3D"bottom" style=3D"width:89.0pt;borde=
r-top:none;border-left:none;border-bottom:solid windowtext 1.0pt;border-rig=
ht:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:15.0pt;border-=
color:currentColor windowtext windowtext currentColor">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">6</span><o:p></o:p></p>
</td>
<td width=3D"169" nowrap=3D"" valign=3D"bottom" style=3D"width:127.0pt;bord=
er-top:none;border-left:none;border-bottom:solid windowtext 1.0pt;border-ri=
ght:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:15.0pt;border=
-color:currentColor windowtext windowtext currentColor">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">8</span><o:p></o:p></p>
</td>
</tr>
<tr style=3D"height:15.0pt">
<td width=3D"153" nowrap=3D"" valign=3D"bottom" style=3D"width:115.0pt;bord=
er:solid windowtext 1.0pt;border-top:none;padding:0in 5.4pt 0in 5.4pt;heigh=
t:15.0pt;border-color:currentColor windowtext windowtext">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">25</span><o:p></o:p></p>
</td>
<td width=3D"119" nowrap=3D"" valign=3D"bottom" style=3D"width:89.0pt;borde=
r-top:none;border-left:none;border-bottom:solid windowtext 1.0pt;border-rig=
ht:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:15.0pt;border-=
color:currentColor windowtext windowtext currentColor">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">7</span><o:p></o:p></p>
</td>
<td width=3D"169" nowrap=3D"" valign=3D"bottom" style=3D"width:127.0pt;bord=
er-top:none;border-left:none;border-bottom:solid windowtext 1.0pt;border-ri=
ght:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:15.0pt;border=
-color:currentColor windowtext windowtext currentColor">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">8</span><o:p></o:p></p>
</td>
</tr>
<tr style=3D"height:15.0pt">
<td width=3D"153" nowrap=3D"" valign=3D"bottom" style=3D"width:115.0pt;bord=
er:solid windowtext 1.0pt;border-top:none;padding:0in 5.4pt 0in 5.4pt;heigh=
t:15.0pt;border-color:currentColor windowtext windowtext">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">20</span><o:p></o:p></p>
</td>
<td width=3D"119" nowrap=3D"" valign=3D"bottom" style=3D"width:89.0pt;borde=
r-top:none;border-left:none;border-bottom:solid windowtext 1.0pt;border-rig=
ht:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:15.0pt;border-=
color:currentColor windowtext windowtext currentColor">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">8</span><o:p></o:p></p>
</td>
<td width=3D"169" nowrap=3D"" valign=3D"bottom" style=3D"width:127.0pt;bord=
er-top:none;border-left:none;border-bottom:solid windowtext 1.0pt;border-ri=
ght:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:15.0pt;border=
-color:currentColor windowtext windowtext currentColor">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">8</span><o:p></o:p></p>
</td>
</tr>
<tr style=3D"height:15.0pt">
<td width=3D"153" nowrap=3D"" valign=3D"bottom" style=3D"width:115.0pt;bord=
er:solid windowtext 1.0pt;border-top:none;padding:0in 5.4pt 0in 5.4pt;heigh=
t:15.0pt;border-color:currentColor windowtext windowtext">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">20</span><o:p></o:p></p>
</td>
<td width=3D"119" nowrap=3D"" valign=3D"bottom" style=3D"width:89.0pt;borde=
r-top:none;border-left:none;border-bottom:solid windowtext 1.0pt;border-rig=
ht:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:15.0pt;border-=
color:currentColor windowtext windowtext currentColor">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">8</span><o:p></o:p></p>
</td>
<td width=3D"169" nowrap=3D"" valign=3D"bottom" style=3D"width:127.0pt;bord=
er-top:none;border-left:none;border-bottom:solid windowtext 1.0pt;border-ri=
ght:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:15.0pt;border=
-color:currentColor windowtext windowtext currentColor">
<p class=3D"MsoNormal" align=3D"center" style=3D"mso-margin-top-alt:auto;ms=
o-margin-bottom-alt:auto;text-align:center">
<span style=3D"color:black">4</span><o:p></o:p></p>
</td>
</tr>
</tbody>
</table>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"> <o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"> <o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto">Thanks!<o:p></o:p></p>
</div>
</div>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"><br>
_________________________________________________________________________<b=
r>
<br>
The information transmitted is intended only for the person or entity to<br=
>
which it is addressed and may contain confidential and/or privileged<br>
material. Any review, retransmission, dissemination or other use of, or<br>
taking of any action in reliance upon, this information by persons or<br>
entities other than the intended recipient is prohibited. If you received<b=
r>
this in error, please contact the sender and delete the material from any<b=
r>
computer.<br>
_________________________________________________________________________<b=
r>
<br>
_______________________________________________<br>
discuss mailing list     <a href=3D"mailto:discuss at mpich.org=
" target=3D"_blank">discuss at mpich.org</a><br>
To manage subscription options or unsubscribe:<br>
<a href=3D"https://lists.mpich.org/mailman/listinfo/discuss" target=3D"_bla=
nk">https://lists.mpich.org/mailman/listinfo/discuss</a><o:p></o:p></p>
</blockquote>
</div>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"> <o:p></o:p></p>
</div>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto"><br>
_________________________________________________________________________<b=
r>
<br>
The information transmitted is intended only for the person or entity to<br=
>
which it is addressed and may contain confidential and/or privileged<br>
material. Any review, retransmission, dissemination or other use of, or<br>
taking of any action in reliance upon, this information by persons or<br>
entities other than the intended recipient is prohibited. If you received<b=
r>
this in error, please contact the sender and delete the material from any<b=
r>
computer.<br>
_________________________________________________________________________<o=
:p></o:p></p>
</div>
</div>
</div>
</div>
<div>
<div>
<p class=3D"MsoNormal"><br>
_________________________________________________________________________<b=
r>
<br>
The information transmitted is intended only for the person or entity to<br=
>
which it is addressed and may contain confidential and/or privileged<br>
material. Any review, retransmission, dissemination or other use of, or<br>
taking of any action in reliance upon, this information by persons or<br>
entities other than the intended recipient is prohibited. If you received<b=
r>
this in error, please contact the sender and delete the material from any<b=
r>
computer.<br>
_________________________________________________________________________<o=
:p></o:p></p>
</div>
</div>
<p class=3D"MsoNormal"><br>
_______________________________________________<br>
discuss mailing list     <a href=3D"mailto:discuss at mpich.org=
">discuss at mpich.org</a><br>
To manage subscription options or unsubscribe:<br>
<a href=3D"https://lists.mpich.org/mailman/listinfo/discuss" target=3D"_bla=
nk">https://lists.mpich.org/mailman/listinfo/discuss</a><o:p></o:p></p>
</blockquote>
</div>
<p class=3D"MsoNormal"><br>
<br clear=3D"all">
<o:p></o:p></p>
<div>
<p class=3D"MsoNormal"><o:p> </o:p></p>
</div>
<p class=3D"MsoNormal">-- <br>
/ Ruben FAELENS<o:p></o:p></p>
<div>
<p class=3D"MsoNormal">+32 494 06 72 59<o:p></o:p></p>
</div>
</div>
</div>
</body>
</html>

<br>
______________________________<wbr>______________________________<wbr>_____=
________<br><br>The information transmitted is intended only for the person=
 or entity to<br>which it is addressed and may contain confidential and/or =
privileged<br>material.  Any review, retransmission, dissemination or other=
 use of, or<br>taking of any action in reliance upon, this information by p=
ersons or<br>entities other than the intended recipient is prohibited.   If=
 you received<br>this in error, please contact the sender and delete the ma=
terial from any<br>computer.<br>______________________________<wbr>________=
______________________<wbr>_____________<br>=

--_000_9D079B25EA4E3149B260846F07AE80754C5944A3TCIEXCH03ustrin_--

--===============3243062315966390760==
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
discuss mailing list     discuss at mpich.org
To manage subscription options or unsubscribe:
https://lists.mpich.org/mailman/listinfo/discuss
--===============3243062315966390760==--

More information about the discuss mailing list