<meta http-equiv="Content-Type" content="text/html; charset=utf-8"><div dir="ltr">MPI_Waitsome also works, and I assume you can use that almost anywhere that MPI_Waitany is used.<div><br></div><div>I didn't determine for sure if it is the cause, but MPI_Waitall, MPI_Wait, and MPI_Waitsome all invoke MPI_Grequest progress, whereas MPI_Waitany does not, and I recall that ROMIO uses generalized requests.</div><div>
<p class="gmail-p1"><span class="gmail-s1">$ grep MPIR_Grequest ../src/mpi/pt2pt/wait*</span></p>
<p class="gmail-p1"><span class="gmail-s2">../src/mpi/pt2pt/waitall.c</span><span class="gmail-s3">:</span><span class="gmail-s1"><span class="gmail-Apple-converted-space"> </span>mpi_errno = </span><span class="gmail-s4"><b>MPIR_Grequest</b></span><span class="gmail-s1">_waitall(count, request_ptrs);</span></p>
<p class="gmail-p1"><span class="gmail-s2">../src/mpi/pt2pt/wait.c</span><span class="gmail-s3">:</span><span class="gmail-s1"><span class="gmail-Apple-tab-span"> </span><span class="gmail-Apple-converted-space"> </span>mpi_errno = </span><span class="gmail-s4"><b>MPIR_Grequest</b></span><span class="gmail-s1">_progress_poke(1, &request_ptr, status);</span></p>
<p class="gmail-p1"><span class="gmail-s2">../src/mpi/pt2pt/waitsome.c</span><span class="gmail-s3">:</span><span class="gmail-s1"><span class="gmail-Apple-tab-span"> </span>mpi_errno = </span><span class="gmail-s4"><b>MPIR_Grequest</b></span><span class="gmail-s1">_progress_poke(incount,<span class="gmail-Apple-converted-space"> </span></span></p><div><br></div><div>Jeff</div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Sep 28, 2017 at 9:32 AM, Jeff Hammond <span dir="ltr"><<a href="mailto:jeff.science@gmail.com" target="_blank">jeff.science@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><br></div><div>There appears to be a bug in Waitany here. It works with 2x Wait and Waitall.</div><div><br></div><div>Jeff</div><div><br></div>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">jrhammon@klondike:~/Work/<wbr>INTEL/BUGS$ mpicc mpiio.c && ./a.out</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">blocking</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">done</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">jrhammon@klondike:~/Work/<wbr>INTEL/BUGS$ mpicc -DNONBLOCKING -DWAITALL mpiio.c && ./a.out</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">nonblocking</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">waitall</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">done</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">jrhammon@klondike:~/Work/<wbr>INTEL/BUGS$ mpicc -DNONBLOCKING -DWAIT mpiio.c && ./a.out</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">nonblocking</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">wait</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">done</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">wait</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">done</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">jrhammon@klondike:~/Work/<wbr>INTEL/BUGS$ mpicc -DNONBLOCKING -DWAITANY mpiio.c && ./a.out</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">nonblocking</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">waitany</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">^C</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">jrhammon@klondike:~/Work/<wbr>INTEL/BUGS$ cat mpiio.c</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">#include <stdio.h></span></p><span class="">
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">#include <mpi.h></span></p>
<p class="m_3986385018240853627gmail-p2"><span class="m_3986385018240853627gmail-s1"></span><br></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">int main(int argc, char** argv)</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">{</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>MPI_Init(&argc, &argv);</span></p>
<p class="m_3986385018240853627gmail-p2"><span class="m_3986385018240853627gmail-s1"></span><br></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>int rank;</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>MPI_Comm_rank(MPI_COMM_WORLD, &rank);</span></p>
<p class="m_3986385018240853627gmail-p2"><span class="m_3986385018240853627gmail-s1"></span><br></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>MPI_Request recv_req;</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>int int_buf;</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>MPI_Irecv(&int_buf, 1, MPI_INT, MPI_ANY_SOURCE, 0, MPI_COMM_WORLD,</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">&recv_req);</span></p>
<p class="m_3986385018240853627gmail-p2"><span class="m_3986385018240853627gmail-s1"></span><br></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>MPI_File fh;</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>MPI_File_open(MPI_COMM_WORLD, "output", MPI_MODE_CREATE | MPI_MODE_RDWR, MPI_INFO_NULL, &fh);</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>MPI_File_set_view(fh, 2*sizeof(int)*rank, MPI_INT, MPI_INT, "native", MPI_INFO_NULL);</span></p>
<p class="m_3986385018240853627gmail-p2"><span class="m_3986385018240853627gmail-s1"></span><br></p>
</span><p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">#ifdef NONBLOCKING</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>printf("nonblocking\n");</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>MPI_Request requests[2] = {MPI_REQUEST_NULL, MPI_REQUEST_NULL};</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>MPI_File_iwrite(fh, &rank, 1, MPI_INT, &requests[0]);</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>MPI_File_iwrite(fh, &rank, 1, MPI_INT, &requests[1]);</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">#if defined(WAITANY)</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>int index;</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>printf("waitany\n");</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>MPI_Waitany(2, requests, &index, MPI_STATUS_IGNORE);</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>MPI_Waitany(2, requests, &index, MPI_STATUS_IGNORE);</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>printf("done\n");</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">#elif defined(WAIT)</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>printf("wait\n");</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>MPI_Wait(&requests[0], MPI_STATUS_IGNORE);</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>printf("done\n");</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>printf("wait\n");</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>MPI_Wait(&requests[1], MPI_STATUS_IGNORE);</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>printf("done\n");</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">#elif defined(WAITALL)</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>printf("waitall\n");</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>MPI_Waitall(2, requests, MPI_STATUSES_IGNORE);</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>printf("done\n");</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">#else</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">#error Define WAITANY, WAIT, or WAITALL</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">#endif</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">#else</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>printf("blocking\n");</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>MPI_File_write(fh, &rank, 1, MPI_INT, MPI_STATUS_IGNORE);</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>MPI_File_write(fh, &rank, 1, MPI_INT, MPI_STATUS_IGNORE);</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>printf("done\n");</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">#endif</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>MPI_File_close(&fh);</span></p>
<p class="m_3986385018240853627gmail-p2"><span class="m_3986385018240853627gmail-s1"></span><br></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>MPI_Finalize();</span></p>
<p class="m_3986385018240853627gmail-p2"><span class="m_3986385018240853627gmail-s1"></span><br></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1"><span class="m_3986385018240853627gmail-Apple-converted-space"> </span>return 0;</span></p>
<p class="m_3986385018240853627gmail-p1"><span class="m_3986385018240853627gmail-s1">}</span></p>
<p class="m_3986385018240853627gmail-p2"><span class="m_3986385018240853627gmail-s1"></span><br></p><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="h5">On Thu, Sep 28, 2017 at 6:53 AM, Nils-Arne Dreier <span dir="ltr"><<a href="mailto:n.dreier@uni-muenster.de" target="_blank">n.dreier@uni-muenster.de</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div class="h5">Dear MPICH community,<br>
<br>
I'm currently playing around with the File-IO interface of MPI. For any<br>
reason, the following code does not running through:<br>
<br>
#include <iostream><br>
#include <mpi.h><br>
<br>
int main(int argc, char** argv)<br>
{<br>
MPI_Init(&argc, &argv);<br>
<br>
int rank;<br>
MPI_Comm_rank(MPI_COMM_WORLD, &rank);<br>
<br>
MPI_Request recv_req;<br>
int int_buf;<br>
MPI_Irecv(&int_buf, 1, MPI_INT, MPI_ANY_SOURCE, 0, MPI_COMM_WORLD,<br>
&recv_req);<br>
<br>
MPI_File fh;<br>
MPI_File_open(MPI_COMM_WORLD, "output", MPI_MODE_CREATE | MPI_MODE_RDWR,<br>
MPI_INFO_NULL, &fh);<br>
MPI_File_set_view(fh, 2*sizeof(int)*rank, MPI_INT, MPI_INT,<br>
"native", MPI_INFO_NULL);<br>
<br>
MPI_Request file_req;<br>
MPI_File_iwrite(fh, &rank, 1, MPI_INT, &file_req);<br>
<br>
MPI_Request file_req2;<br>
MPI_File_iwrite(fh, &rank, 1, MPI_INT, &file_req2);<br>
<br>
MPI_Request requests[2] = {file_req, file_req2};<br>
int index;<br>
std::cout << rank << ":\twaiting..." << std::endl;<br>
MPI_Waitany(2, requests, &index, MPI_STATUS_IGNORE);<br>
//MPI_Wait(&file_req, MPI_STATUS_IGNORE);<br>
std::cout << rank << ":\tdone" << std::endl;<br>
MPI_File_close(&fh);<br>
<br>
MPI_Finalize();<br>
<br>
return 0;<br>
}<br>
<br>
Neither if i exchange file_req2 with recv_req, MPI_Waitany doesn't return.<br>
<br>
Is it possible to use MPI_Waitany with File-IO calls? Did i read<br>
anything over in the MPI-Standard?<br>
<br>
Thank you for your help.<br>
<br>
Thanks,<br>
Nils<br>
<br>
PS: I'm running Ubuntu 16.04 LTS, gcc-4.9 (using -std=c++14) and MPICH<br>
3.2. This is the output of mpirun --version:<br>
HYDRA build details:<br>
Version: <wbr> 3.2<br>
Release Date: <wbr> Wed Nov 11 22:06:48 CST 2015<br>
CC: <wbr> gcc -Wl,-Bsymbolic-functions<br>
-Wl,-z,relro<br>
CXX: <wbr> g++ -Wl,-Bsymbolic-functions<br>
-Wl,-z,relro<br>
F77: <wbr> gfortran -Wl,-Bsymbolic-functions<br>
-Wl,-z,relro<br>
F90: <wbr> gfortran -Wl,-Bsymbolic-functions<br>
-Wl,-z,relro<br>
Configure options: '--disable-option-checking'<br>
'--prefix=/usr' '--build=x86_64-linux-gnu'<br>
'--includedir=${prefix}/includ<wbr>e' '--mandir=${prefix}/share/man'<br>
'--infodir=${prefix}/share/inf<wbr>o' '--sysconfdir=/etc'<br>
'--localstatedir=/var' '--disable-silent-rules'<br>
'--libdir=${prefix}/lib/x86_64<wbr>-linux-gnu'<br>
'--libexecdir=${prefix}/lib/x8<wbr>6_64-linux-gnu'<br>
'--disable-maintainer-mode' '--disable-dependency-tracking<wbr>'<br>
'--enable-shared' '--enable-fortran=all' '--disable-rpath'<br>
'--disable-wrapper-rpath' '--sysconfdir=/etc/mpich'<br>
'--libdir=/usr/lib/x86_64-linu<wbr>x-gnu' '--includedir=/usr/include/mpi<wbr>ch'<br>
'--docdir=/usr/share/doc/mpich<wbr>' '--with-hwloc-prefix=system'<br>
'--enable-checkpointing' '--with-hydra-ckpointlib=blcr' 'CPPFLAGS=<br>
-Wdate-time -D_FORTIFY_SOURCE=2<br>
-I/build/mpich-jQtQ8p/mpich-3.<wbr>2/src/mpl/include<br>
-I/build/mpich-jQtQ8p/mpich-3.<wbr>2/src/mpl/include<br>
-I/build/mpich-jQtQ8p/mpich-3.<wbr>2/src/openpa/src<br>
-I/build/mpich-jQtQ8p/mpich-3.<wbr>2/src/openpa/src -D_REENTRANT<br>
-I/build/mpich-jQtQ8p/mpich-3.<wbr>2/src/mpi/romio/include' 'CFLAGS= -g -O2<br>
-fstack-protector-strong -Wformat -Werror=format-security -O2'<br>
'CXXFLAGS= -g -O2 -fstack-protector-strong -Wformat<br>
-Werror=format-security -O2' 'FFLAGS= -g -O2 -fstack-protector-strong<br>
-O2' 'FCFLAGS= -g -O2 -fstack-protector-strong -O2'<br>
'build_alias=x86_64-linux-gnu' 'MPICHLIB_CFLAGS=-g -O2<br>
-fstack-protector-strong -Wformat -Werror=format-security'<br>
'MPICHLIB_CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'<br>
'MPICHLIB_CXXFLAGS=-g -O2 -fstack-protector-strong -Wformat<br>
-Werror=format-security' 'MPICHLIB_FFLAGS=-g -O2<br>
-fstack-protector-strong' 'MPICHLIB_FCFLAGS=-g -O2<br>
-fstack-protector-strong' 'LDFLAGS=-Wl,-Bsymbolic-functi<wbr>ons<br>
-Wl,-z,relro' 'FC=gfortran' 'F77=gfortran' 'MPILIBNAME=mpich'<br>
'--cache-file=/dev/null' '--srcdir=.' 'CC=gcc' 'LIBS=-lpthread '<br>
Process Manager: <wbr> pmi<br>
Launchers available: ssh rsh fork slurm ll lsf<br>
sge manual persist<br>
Topology libraries available: hwloc<br>
Resource management kernels available: user slurm ll lsf sge pbs<br>
cobalt<br>
Checkpointing libraries available: blcr<br>
Demux engines available: poll select<br>
<span class="m_3986385018240853627gmail-HOEnZb"><font color="#888888"><br>
<br>
--<br>
Nils-Arne Dreier, M.Sc.<br>
Institute for Computational und Applied Mathematics,<br>
University of Münster, Orleans-Ring 10, D-48149 Münster<br>
Tel: +49 251 83-35147<br>
<br>
</font></span><br></div></div>______________________________<wbr>_________________<br>
discuss mailing list <a href="mailto:discuss@mpich.org" target="_blank">discuss@mpich.org</a><br>
To manage subscription options or unsubscribe:<br>
<a href="https://lists.mpich.org/mailman/listinfo/discuss" rel="noreferrer" target="_blank">https://lists.mpich.org/mailma<wbr>n/listinfo/discuss</a><span class="HOEnZb"><font color="#888888"><br></font></span></blockquote></div><span class="HOEnZb"><font color="#888888"><br><br clear="all"><div><br></div>-- <br><div class="m_3986385018240853627gmail_signature">Jeff Hammond<br><a href="mailto:jeff.science@gmail.com" target="_blank">jeff.science@gmail.com</a><br><a href="http://jeffhammond.github.io/" target="_blank">http://jeffhammond.github.io/</a></div>
</font></span></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature">Jeff Hammond<br><a href="mailto:jeff.science@gmail.com" target="_blank">jeff.science@gmail.com</a><br><a href="http://jeffhammond.github.io/" target="_blank">http://jeffhammond.github.io/</a></div>
</div>