<div dir="ltr"><div>Thanks a lot! Jim. Now I make sense<br><br></div>Sufeng<br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Jul 5, 2013 at 11:25 AM, <span dir="ltr"><<a href="mailto:discuss-request@mpich.org" target="_blank">discuss-request@mpich.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Send discuss mailing list submissions to<br>
<a href="mailto:discuss@mpich.org">discuss@mpich.org</a><br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="https://lists.mpich.org/mailman/listinfo/discuss" target="_blank">https://lists.mpich.org/mailman/listinfo/discuss</a><br>
or, via email, send a message with subject or body 'help' to<br>
<a href="mailto:discuss-request@mpich.org">discuss-request@mpich.org</a><br>
<br>
You can reach the person managing the list at<br>
<a href="mailto:discuss-owner@mpich.org">discuss-owner@mpich.org</a><br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of discuss digest..."<br>
<br>
<br>
Today's Topics:<br>
<br>
1. Re: beginner for remote mem access (sufeng) (Pavan Balaji)<br>
2. MPI Benchmark Suite (Jiri Simsa)<br>
3. Re: beginner for remote mem access (sufeng) (Jim Dinan)<br>
4. Re: beginner for remote mem access (sufeng) (Sufeng Niu)<br>
5. Re: beginner for remote mem access (sufeng) (Pavan Balaji)<br>
6. Re: MPI Benchmark Suite (Pavan Balaji)<br>
7. Re: beginner for remote mem access (sufeng) (Sufeng Niu)<br>
<br>
<br>
----------------------------------------------------------------------<br>
<br>
Message: 1<br>
Date: Fri, 05 Jul 2013 10:36:07 -0500<br>
From: Pavan Balaji <<a href="mailto:balaji@mcs.anl.gov">balaji@mcs.anl.gov</a>><br>
To: <a href="mailto:discuss@mpich.org">discuss@mpich.org</a><br>
Cc: Sufeng Niu <<a href="mailto:sniu@hawk.iit.edu">sniu@hawk.iit.edu</a>><br>
Subject: Re: [mpich-discuss] beginner for remote mem access (sufeng)<br>
Message-ID: <<a href="mailto:51D6E7E7.8070306@mcs.anl.gov">51D6E7E7.8070306@mcs.anl.gov</a>><br>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed<br>
<br>
<br>
On 07/05/2013 10:28 AM, Sufeng Niu wrote:<br>
> 1.<br>
> MPI_Win_fence is not similar to MPI_Barrier in some cases. The<br>
> MPICH implementation can turn some calls into no-ops and others into<br>
> MPI_Reduce_scatter.<br>
> I am not quite sure about "turn some calls into no-ops and others into<br>
> MPI_Reduce_scatter" Could you please give an example if possible?<br>
> Another thing is if a process create a window for other process, but the<br>
> data access is available after some operations. Should I use<br>
> MPI_Win_fence or MPI_Barrier to sync? or other methods?<br>
<br>
MPI_WIN_FENCE will make sure that all PUT/GET/ACCUMULATE operations are<br>
complete locally and remotely. This is probably what you need.<br>
<br>
MPI_BARRIER is useless in this case since PUT/GET/ACCUMULATE can be<br>
nonblocking.<br>
<br>
> 2.<br>
> Regarding "use MPI window as thread level", I really don't<br>
> understand your thinking at all. MPI RMA is not shared memory nor is<br>
> MPI a threading model.<br>
> Sorry for uncleared statements, right now I would like to do<br>
> multithreads and MPI hybrid programming. let me give an example: I have<br>
> 3 processes, each one has 8 threads. thread 0 in process 0 creates a RMA<br>
> window. if I would like all other threads to access it, should I use<br>
> thread 0 in process 1, thread 0 in process 2 to MPI_Get the data from<br>
> window, then use shared memory for internal threads to load the data? i<br>
> am not sure what is the proper way for RMA in hybrid model.<br>
<br>
All threads within the OS process are part of the same MPI rank. Any of<br>
them can access the window as the same rank.<br>
<br>
-- Pavan<br>
<br>
--<br>
Pavan Balaji<br>
<a href="http://www.mcs.anl.gov/~balaji" target="_blank">http://www.mcs.anl.gov/~balaji</a><br>
<br>
<br>
------------------------------<br>
<br>
Message: 2<br>
Date: Fri, 5 Jul 2013 11:38:13 -0400<br>
From: Jiri Simsa <<a href="mailto:jsimsa@cs.cmu.edu">jsimsa@cs.cmu.edu</a>><br>
To: <a href="mailto:discuss@mpich.org">discuss@mpich.org</a><br>
Subject: [mpich-discuss] MPI Benchmark Suite<br>
Message-ID:<br>
<CAHs9ut-=<a href="mailto:FJeUDG-T2-qqDhcDMBGfGuXPHG7VQp8_xYzXTxPAzA@mail.gmail.com">FJeUDG-T2-qqDhcDMBGfGuXPHG7VQp8_xYzXTxPAzA@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="iso-8859-1"<br>
<br>
Hello,<br>
<br>
Could anyone point me in the direction of programs that are representative<br>
of MPI programs? I am looking for something like PARSEC (<br>
<a href="http://parsec.cs.princeton.edu/" target="_blank">http://parsec.cs.princeton.edu/</a>) for MPI. In other words, I am interested<br>
in macro-bechnmarks not micro-benchmarks. Thank you.<br>
<br>
Best,<br>
<br>
--Jiri Simsa<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://lists.mpich.org/pipermail/discuss/attachments/20130705/ab3a6a55/attachment-0001.html" target="_blank">http://lists.mpich.org/pipermail/discuss/attachments/20130705/ab3a6a55/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 3<br>
Date: Fri, 5 Jul 2013 11:43:29 -0400<br>
From: Jim Dinan <<a href="mailto:james.dinan@gmail.com">james.dinan@gmail.com</a>><br>
To: <a href="mailto:discuss@mpich.org">discuss@mpich.org</a><br>
Subject: Re: [mpich-discuss] beginner for remote mem access (sufeng)<br>
Message-ID:<br>
<<a href="mailto:CAOoEU4E1j4x7hxR6FrZMLrjsfaH31tzDTgucyiF5F3osduxY2Q@mail.gmail.com">CAOoEU4E1j4x7hxR6FrZMLrjsfaH31tzDTgucyiF5F3osduxY2Q@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="iso-8859-1"<br>
<br>
On Fri, Jul 5, 2013 at 11:28 AM, Sufeng Niu <<a href="mailto:sniu@hawk.iit.edu">sniu@hawk.iit.edu</a>> wrote:<br>
<br>
> 1.<br>
> MPI_Win_fence is not similar to MPI_Barrier in some cases. The<br>
> MPICH implementation can turn some calls into no-ops and others into<br>
> MPI_Reduce_scatter.<br>
> I am not quite sure about "turn some calls into no-ops and others into<br>
> MPI_Reduce_scatter" Could you please give an example if possible? Another<br>
> thing is if a process create a window for other process, but the data<br>
> access is available after some operations. Should I use MPI_Win_fence or<br>
> MPI_Barrier to sync? or other methods?<br>
><br>
<br>
When you call MPI_Win_fence, the MPI implementation does not need to wait<br>
for all processes to call the function before proceeding. It does require<br>
that all RMA operations in a given process' access epoch are complete, and<br>
that all RMA accesses in its exposure epoch (i.e., targeting the given<br>
process) are also complete. If, for example, there is no preceding epoch,<br>
the call to MPI_Win_fence is a no op.<br>
<br>
Generally speaking, you should synchronize all accesses to data exposed in<br>
a window using RMA synchronization operations. Barrier does not provide<br>
any consistency for RMA operations.<br>
<br>
<br>
> 2.<br>
> Regarding "use MPI window as thread level", I really don't<br>
> understand your thinking at all. MPI RMA is not shared memory nor is<br>
> MPI a threading model.<br>
> Sorry for uncleared statements, right now I would like to do multithreads<br>
> and MPI hybrid programming. let me give an example: I have 3 processes,<br>
> each one has 8 threads. thread 0 in process 0 creates a RMA window. if I<br>
> would like all other threads to access it, should I use thread 0 in process<br>
> 1, thread 0 in process 2 to MPI_Get the data from window, then use shared<br>
> memory for internal threads to load the data? i am not sure what is the<br>
> proper way for RMA in hybrid model.<br>
><br>
<br>
The threads can all access the data in the window, provided that the MPI<br>
process synchronizes accesses by threads in the process correctly with<br>
respect to what other processes are doing. Note that data consistency<br>
across threads is outside of the MPI specification, and you should use e.g.<br>
Pthreads or OpenMP primitives to ensure that data is consistent across<br>
threads.<br>
<br>
~Jim.<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://lists.mpich.org/pipermail/discuss/attachments/20130705/d2064a55/attachment-0001.html" target="_blank">http://lists.mpich.org/pipermail/discuss/attachments/20130705/d2064a55/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 4<br>
Date: Fri, 5 Jul 2013 11:18:22 -0500<br>
From: Sufeng Niu <<a href="mailto:sniu@hawk.iit.edu">sniu@hawk.iit.edu</a>><br>
To: Pavan Balaji <<a href="mailto:balaji@mcs.anl.gov">balaji@mcs.anl.gov</a>><br>
Cc: <a href="mailto:discuss@mpich.org">discuss@mpich.org</a><br>
Subject: Re: [mpich-discuss] beginner for remote mem access (sufeng)<br>
Message-ID:<br>
<<a href="mailto:CAFNNHkyV3k1168kYjOhX2OpeF%2BTcjLRsg6Oj2oL0pZ-KwHQKXw@mail.gmail.com">CAFNNHkyV3k1168kYjOhX2OpeF+TcjLRsg6Oj2oL0pZ-KwHQKXw@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="iso-8859-1"<br>
<br>
Hi Pavan,<br>
<br>
Thanks a lot!<br>
<br>
Now I make sense for MPI_Win_fence.<br>
<br>
for hybrid RMA issue, did you mean in each rank, remote get the data into a<br>
shared memory, then threads in each rank load it by thread-level operation?<br>
<br>
Thank you!<br>
Sufeng<br>
<br>
<br>
<br>
On Fri, Jul 5, 2013 at 10:36 AM, Pavan Balaji <<a href="mailto:balaji@mcs.anl.gov">balaji@mcs.anl.gov</a>> wrote:<br>
<br>
><br>
> On 07/05/2013 10:28 AM, Sufeng Niu wrote:<br>
><br>
>> 1.<br>
>> MPI_Win_fence is not similar to MPI_Barrier in some cases. The<br>
>> MPICH implementation can turn some calls into no-ops and others into<br>
>> MPI_Reduce_scatter.<br>
>> I am not quite sure about "turn some calls into no-ops and others into<br>
>> MPI_Reduce_scatter" Could you please give an example if possible?<br>
>> Another thing is if a process create a window for other process, but the<br>
>> data access is available after some operations. Should I use<br>
>> MPI_Win_fence or MPI_Barrier to sync? or other methods?<br>
>><br>
><br>
> MPI_WIN_FENCE will make sure that all PUT/GET/ACCUMULATE operations are<br>
> complete locally and remotely. This is probably what you need.<br>
><br>
> MPI_BARRIER is useless in this case since PUT/GET/ACCUMULATE can be<br>
> nonblocking.<br>
><br>
><br>
> 2.<br>
>> Regarding "use MPI window as thread level", I really don't<br>
>> understand your thinking at all. MPI RMA is not shared memory nor is<br>
>> MPI a threading model.<br>
>> Sorry for uncleared statements, right now I would like to do<br>
>> multithreads and MPI hybrid programming. let me give an example: I have<br>
>> 3 processes, each one has 8 threads. thread 0 in process 0 creates a RMA<br>
>> window. if I would like all other threads to access it, should I use<br>
>> thread 0 in process 1, thread 0 in process 2 to MPI_Get the data from<br>
>> window, then use shared memory for internal threads to load the data? i<br>
>> am not sure what is the proper way for RMA in hybrid model.<br>
>><br>
><br>
> All threads within the OS process are part of the same MPI rank. Any of<br>
> them can access the window as the same rank.<br>
><br>
> -- Pavan<br>
><br>
> --<br>
> Pavan Balaji<br>
> <a href="http://www.mcs.anl.gov/~balaji" target="_blank">http://www.mcs.anl.gov/~balaji</a><br>
><br>
<br>
<br>
<br>
--<br>
Best Regards,<br>
Sufeng Niu<br>
ECASP lab, ECE department, Illinois Institute of Technology<br>
Tel: <a href="tel:312-731-7219" value="+13127317219">312-731-7219</a><br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://lists.mpich.org/pipermail/discuss/attachments/20130705/2454c8c0/attachment-0001.html" target="_blank">http://lists.mpich.org/pipermail/discuss/attachments/20130705/2454c8c0/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 5<br>
Date: Fri, 05 Jul 2013 11:20:41 -0500<br>
From: Pavan Balaji <<a href="mailto:balaji@mcs.anl.gov">balaji@mcs.anl.gov</a>><br>
To: Sufeng Niu <<a href="mailto:sniu@hawk.iit.edu">sniu@hawk.iit.edu</a>><br>
Cc: <a href="mailto:discuss@mpich.org">discuss@mpich.org</a><br>
Subject: Re: [mpich-discuss] beginner for remote mem access (sufeng)<br>
Message-ID: <<a href="mailto:51D6F259.80800@mcs.anl.gov">51D6F259.80800@mcs.anl.gov</a>><br>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed<br>
<br>
<br>
On 07/05/2013 11:18 AM, Sufeng Niu wrote:<br>
> for hybrid RMA issue, did you mean in each rank, remote get the data<br>
> into a shared memory, then threads in each rank load it by thread-level<br>
> operation?<br>
<br>
All threads are in the same process, so they all already share memory.<br>
Perhaps its best if you try to learn how threads work before attempting<br>
to use them with MPI.<br>
<br>
-- Pavan<br>
<br>
--<br>
Pavan Balaji<br>
<a href="http://www.mcs.anl.gov/~balaji" target="_blank">http://www.mcs.anl.gov/~balaji</a><br>
<br>
<br>
------------------------------<br>
<br>
Message: 6<br>
Date: Fri, 05 Jul 2013 11:22:28 -0500<br>
From: Pavan Balaji <<a href="mailto:balaji@mcs.anl.gov">balaji@mcs.anl.gov</a>><br>
To: <a href="mailto:discuss@mpich.org">discuss@mpich.org</a><br>
Cc: Jiri Simsa <<a href="mailto:jsimsa@cs.cmu.edu">jsimsa@cs.cmu.edu</a>><br>
Subject: Re: [mpich-discuss] MPI Benchmark Suite<br>
Message-ID: <<a href="mailto:51D6F2C4.7030002@mcs.anl.gov">51D6F2C4.7030002@mcs.anl.gov</a>><br>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed<br>
<br>
<br>
On 07/05/2013 10:38 AM, Jiri Simsa wrote:<br>
> Could anyone point me in the direction of programs that are<br>
> representative of MPI programs? I am looking for something like PARSEC<br>
> (<a href="http://parsec.cs.princeton.edu/" target="_blank">http://parsec.cs.princeton.edu/</a>) for MPI. In other words, I am<br>
> interested in macro-bechnmarks not micro-benchmarks. Thank you.<br>
<br>
You could try the Sequoia suite. There's also the NAS parallel<br>
benchmarks and Graph500 benchmarks.<br>
<br>
-- Pavan<br>
<br>
--<br>
Pavan Balaji<br>
<a href="http://www.mcs.anl.gov/~balaji" target="_blank">http://www.mcs.anl.gov/~balaji</a><br>
<br>
<br>
------------------------------<br>
<br>
Message: 7<br>
Date: Fri, 5 Jul 2013 11:24:59 -0500<br>
From: Sufeng Niu <<a href="mailto:sniu@hawk.iit.edu">sniu@hawk.iit.edu</a>><br>
To: Pavan Balaji <<a href="mailto:balaji@mcs.anl.gov">balaji@mcs.anl.gov</a>><br>
Cc: <a href="mailto:discuss@mpich.org">discuss@mpich.org</a><br>
Subject: Re: [mpich-discuss] beginner for remote mem access (sufeng)<br>
Message-ID:<br>
<<a href="mailto:CAFNNHkzCaNw%2BgQhSriR1xUOo853bNPAtM7Z3TTOBS_XB1JD3ww@mail.gmail.com">CAFNNHkzCaNw+gQhSriR1xUOo853bNPAtM7Z3TTOBS_XB1JD3ww@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="iso-8859-1"<br>
<br>
Thanks a lot!<br>
<br>
Now I got it.<br>
<br>
Sufeng<br>
<br>
<br>
On Fri, Jul 5, 2013 at 11:20 AM, Pavan Balaji <<a href="mailto:balaji@mcs.anl.gov">balaji@mcs.anl.gov</a>> wrote:<br>
<br>
><br>
> On 07/05/2013 11:18 AM, Sufeng Niu wrote:<br>
><br>
>> for hybrid RMA issue, did you mean in each rank, remote get the data<br>
>> into a shared memory, then threads in each rank load it by thread-level<br>
>> operation?<br>
>><br>
><br>
> All threads are in the same process, so they all already share memory.<br>
> Perhaps its best if you try to learn how threads work before attempting to<br>
> use them with MPI.<br>
><br>
><br>
> -- Pavan<br>
><br>
> --<br>
> Pavan Balaji<br>
> <a href="http://www.mcs.anl.gov/~balaji" target="_blank">http://www.mcs.anl.gov/~balaji</a><br>
><br>
<br>
<br>
<br>
--<br>
Best Regards,<br>
Sufeng Niu<br>
ECASP lab, ECE department, Illinois Institute of Technology<br>
Tel: <a href="tel:312-731-7219" value="+13127317219">312-731-7219</a><br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://lists.mpich.org/pipermail/discuss/attachments/20130705/f0eefadd/attachment.html" target="_blank">http://lists.mpich.org/pipermail/discuss/attachments/20130705/f0eefadd/attachment.html</a>><br>
<br>
------------------------------<br>
<br>
_______________________________________________<br>
discuss mailing list<br>
<a href="mailto:discuss@mpich.org">discuss@mpich.org</a><br>
<a href="https://lists.mpich.org/mailman/listinfo/discuss" target="_blank">https://lists.mpich.org/mailman/listinfo/discuss</a><br>
<br>
End of discuss Digest, Vol 9, Issue 10<br>
**************************************<br>
</blockquote></div><br><br clear="all"><br>-- <br>Best Regards,<div>Sufeng Niu</div><div>ECASP lab, ECE department, Illinois Institute of Technology</div><div>Tel: 312-731-7219</div>
</div>