<div dir="ltr"><div>Thanks. This is helpful.</div><div><br></div><div>In order to avoid ABI issues with the status object, I was thinking instead we could define attributes on requests. That is sufficient to solve all sorts of problems, because one can just get the attributes off the request and decide to clean up as appropriate.</div><div><br></div><div>Do you think adding attributes to requests causes any implementation headaches? My expectation is that it does not slow down any critical path because the only time the attribute fields are touched are when the user accesses them. The internal object behind a request would need an additional field, but that's a small change to MPIR_Request.</div><div><br></div><div>However, if we end up standardizing a 256-bit status object in MPI-5 (hypothetically), then we have an option to put hidden fields for this in status. It's debatable if MPI_Status_get_buffer_address(MPI_Status * status) is significantly better than a request attribute lookup.</div><div><br></div><div>In any case, I figured out that I can do the deferred implementation using generalized requests. It requires Pthreads and MPI_THREAD_MULTIPLE but I can live with that.</div><div><br></div><div>Jeff</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Jan 6, 2023 at 12:39 AM Zhou, Hui <<a href="mailto:zhouh@anl.gov">zhouh@anl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="msg-8463918519245916947">
<div dir="ltr">
<div><span style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">Hi Jeff,</span></div>
<div><span style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgb(255,255,255)"><br>
</span></div>
<div><span style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">The nonblocking cases can also do the same support by copying to and from contiguous buffers
if we stick the temporary buffer pointer in the request object and free it upon completion. But if you are looking at this from the Fortran binding layer, which does not have access to MPICH internals, then it is difficult to do. The straight-forward solution
would be ask MPICH to add an extension, so the user can stick the buffer pointer somewhere, for example, the status object.<br>
</span></div>
<div><span style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgb(255,255,255)"><br>
</span></div>
<blockquote style="border-color:rgb(200,200,200);border-left-width:3px;border-left-style:solid;padding-left:1ex;margin-left:0.8ex;color:rgb(102,102,102)">
<div><span style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">I wonder if there is a code path in MPICH associated with nonblocking
communication that is deferred until the request is synchronized. For example, is there code that merely copies all the arguments passed to the call into some object and then processes the whole operation when the request is synchronized?<br>
</span></div>
</blockquote>
<div>I am not exactly sure what do you mean here, but the persistent request most resembles your idea of merely hold all the arguments and only process it at start time. But again, if you are not allowed to access MPICH internals, then that is out-of-reach.</div>
<div><span><br>
</span></div>
<div><span>-- <br>
</span></div>
<div><span>Hui<br>
</span></div>
<div id="m_-8463918519245916947appendonsend"></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<hr style="display:inline-block;width:98%">
<div id="m_-8463918519245916947divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> Jeff Hammond via devel <<a href="mailto:devel@mpich.org" target="_blank">devel@mpich.org</a>><br>
<b>Sent:</b> Thursday, January 5, 2023 4:50 AM<br>
<b>To:</b> MPICH <<a href="mailto:devel@mpich.org" target="_blank">devel@mpich.org</a>><br>
<b>Cc:</b> Jeff Hammond <<a href="mailto:jeff.science@gmail.com" target="_blank">jeff.science@gmail.com</a>><br>
<b>Subject:</b> [mpich-devel] code path for non-blocking operations that are deferred until synchronization</font>
<div> </div>
</div>
<div>
<div dir="ltr">This is related to <a href="https://github.com/mpi-forum/mpi-issues/issues/663" target="_blank">https://github.com/mpi-forum/mpi-issues/issues/663</a> but there are other scenarios where nonblocking communication with non-contiguous
Fortran subarrays is extremely difficult to implement. The blocking cases are easy to support by copying to and from contiguous buffers.
<div><br>
</div>
<div>I wonder if there is a code path in MPICH associated with nonblocking communication that is deferred until the request is synchronized. For example, is there code that merely copies all the arguments passed to the call into some object and then processes
the whole operation when the request is synchronized?</div>
<div><br>
</div>
<div>Critically for my case, the Fortran arguments would need to be copied, meaning the CFI_cdesc_t pointer not just the associated base_addr, which I doubt is there based on what I've read so far.</div>
<div><br>
Thanks,</div>
<div><br>
</div>
<div>Jeff<br>
<div>
<div><br>
</div>
-- <br>
<div dir="ltr">Jeff Hammond<br>
<a href="mailto:jeff.science@gmail.com" target="_blank">jeff.science@gmail.com</a><br>
<a href="http://jeffhammond.github.io/" target="_blank">http://jeffhammond.github.io/</a></div>
</div>
</div>
</div>
</div>
</div>
</div></blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature">Jeff Hammond<br><a href="mailto:jeff.science@gmail.com" target="_blank">jeff.science@gmail.com</a><br><a href="http://jeffhammond.github.io/" target="_blank">http://jeffhammond.github.io/</a></div></div>