<font size=2 face="sans-serif">I've been out sick and haven't followed
this conversation too closely, but I've gotten a support note that seems
somewhat related -- at least they're asking about external32. Any
comments from the experts before I try to dig in? ROMIO supports
external32? A quick search shows external32 code is common, not ad_bg
specific.</font>
<br>
<br><font size=2 face="sans-serif">---------
</font>
<br><font size=2 face="sans-serif">The MPI standard defines a portable
data representation for MPI I/O, </font>
<br><font size=2 face="sans-serif">see e.g. [1]. On BGQ, attempting to
write data via MPI I/O with </font>
<br><font size=2 face="sans-serif">the data representation set to 'external32',
MPI does not report any </font>
<br><font size=2 face="sans-serif">errors but the data written is not in
the correct format. Furthermore </font>
<br><font size=2 face="sans-serif">trying to read the data back with the
same data representation fails </font>
<br><font size=2 face="sans-serif">and does not reproduce the original
data.
</font>
<br><font size=2 face="sans-serif">
</font>
<br><font size=2 face="sans-serif">The example Fortran code does the following:
</font>
<br><font size=2 face="sans-serif">1) write a few numbers (integer, float
and double) to file via MPI I/O </font>
<br><font size=2 face="sans-serif">in native format
</font>
<br><font size=2 face="sans-serif">2) read the numbers back in native format
</font>
<br><font size=2 face="sans-serif">3) write the same numbers to file in
external32 format
</font>
<br><font size=2 face="sans-serif">4) output MPI errors/success when using
MPI I/O
</font>
<br><font size=2 face="sans-serif">5) read the numbers back from file
</font>
<br><font size=2 face="sans-serif">
</font>
<br><font size=2 face="sans-serif">1+2 work flawlessly
</font>
<br><font size=2 face="sans-serif">3+4 show no errors and instead indicate
MPI SUCCESS
</font>
<br><font size=2 face="sans-serif">5 fails to reproduce correct numbers
</font>
<br><font size=2 face="sans-serif">
</font>
<br><font size=2 face="sans-serif">The data written in 1) appears as big
endian data as expected. The </font>
<br><font size=2 face="sans-serif">data written in 3) seems to be little
endian (for integer and float) </font>
<br><font size=2 face="sans-serif">and strangely byte-swapped for doubles.
</font>
<br><font size=2 face="sans-serif">
</font>
<br><font size=2 face="sans-serif">[1] </font><a href="http://www.mpi-forum.org/docs/mpi21-report/node285.htm"><font size=2 face="sans-serif">http://www.mpi-forum.org/docs/mpi21-report/node285.htm</font></a><font size=2 face="sans-serif">
</font>
<br><font size=2 face="sans-serif">
</font>
<br><font size=2 face="sans-serif"><br>
Bob Cernohous: (T/L 553) 507-253-6093<br>
<br>
BobC@us.ibm.com<br>
IBM Rochester, Building 030-2(C335), Department 61L<br>
3605 Hwy 52 North, Rochester, MN 55901-7829<br>
<br>
> Chaos reigns within.<br>
> Reflect, repent, and reboot.<br>
> Order shall return.<br>
</font>
<br>
<br>
<br>
<br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif">Dave Goodell <goodell@mcs.anl.gov></font>
<br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif">Larry Baker <baker@usgs.gov>,
</font>
<br><font size=1 color=#5f5f5f face="sans-serif">Cc:
</font><font size=1 face="sans-serif">devel@mpich.org</font>
<br><font size=1 color=#5f5f5f face="sans-serif">Date:
</font><font size=1 face="sans-serif">04/02/2013 04:48 PM</font>
<br><font size=1 color=#5f5f5f face="sans-serif">Subject:
</font><font size=1 face="sans-serif">Re: [mpich-devel]
binding a new MPI type with fortran</font>
<br><font size=1 color=#5f5f5f face="sans-serif">Sent by:
</font><font size=1 face="sans-serif">devel-bounces@mpich.org</font>
<br>
<hr noshade>
<br>
<br>
<br><tt><font size=2>On Apr 2, 2013, at 4:34 PM CDT, Larry Baker <baker@usgs.gov>
wrote:<br>
<br>
> Jeff referred me to the MPI 3.0 spec (</font></tt><a href="http://mpi-forum.org/docs/mpi-3.0/mpi30-report.pdf"><tt><font size=2>http://mpi-forum.org/docs/mpi-3.0/mpi30-report.pdf</font></tt></a><tt><font size=2>)
for data sizes. But, even it is contradictory. For example,
in section 3.2.2, it says an MPI_INTEGER is a Fortran (default KIND) INTEGER.
Fortran does not specify the size of a (default KIND) INTEGER. However,
Table 13.2 says it is 4 bytes. It could just as easily be 8 bytes
in a 64-bit execution environment. In that case, Fortran requires
that the size of all the default KIND numeric types in Table 13.2 would
double in size. But, not necessarily pointers; pointers in Fortran
do not have a specified size (storage unit). <br>
<br>
Table 13.2 lists sizes for a specific kind of encoding of MPI datatypes
known as "external32". You should not use it as a reference
for the actual size of the MPI datatype, which must instead always match
the actual language type in the corresponding language binding.<br>
<br>
> The MPI 3.0 spec says if you send an MPI_INTEGER, you have to receive
an MPI_INTEGER. I didn't exhaustively search the MPI 3.0 spec, but
you might also have to make sure the default KIND INTEGER is the same for
the sender and receiver. This might be beyond the scope of MPI, and
I do not know how exhaustive the run-time tests can be to detect violations
of the rules. (A message would have to carry type metadata for the
payload data in addition to the payload data itself.) Section 17
had lots of information about Fortran and C data sizes, including automatically
conversions. I didn't read enough to know if the spec covers the
case of exchanging default KIND INTEGERs between two executables with different
sizes. Maybe it is true that MPI mandates something like lexical
scoping rules, such that, for example, all members of an MPI application
must assign the same semantics to their fundamental data types -- the default
KIND INTEGER, etc., in the case of Fortran.<br>
<br>
Most users should ignore the rules regarding conversions on heterogeneous
systems. They don't make a lot of sense even if you're an MPI expert,
and:<br>
<br>
1) almost no MPI implementations properly support this behavior<br>
<br>
2) almost no user codes are written correctly to deal with it, even if
#1 were true<br>
<br>
-Dave<br>
</font></tt>
<br>