<div dir="ltr">MPI does not know about the padding because it does not really create a struct in the same sense as the compiler. As a result, you need to resize the datatype to account for the padding. Check the examples 5.16 and 5.17 in the MPI standard to see how to use it.<div><br></div><div>For your particular example, replacing the last 2 lines of code with:</div><div><br></div><div>MPI_Type_create_struct(struct_len, block_len, displacements, types, &tmp_ddt);<br></div><div>MPI_Type_create_resized(tmp_ddt, 0, sizeof(), &mpi_data_type);</div><div>MPI_Type_connit(mpi_data_type);</div><div><br></div><div>should make the MPI struct match the compiler struct.</div><div><br></div><div>While you can manipulate the pointers directly and compute the displacements by hand, you should not use a size_t for that. MPI_Aint is a signed type, ssize_t or some form of ptrdiff_t. Instead you should use the MPI suggested way via MPI_Get_address (to obtain the address of an object as an MPI_Aint), and then MPI_Aint_diff to compute relative displacements.</div><div><br></div><div> George.</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Dec 3, 2023 at 3:55 PM Mccall, Kurt E. (MSFC-EV41) via discuss <<a href="mailto:discuss@mpich.org">discuss@mpich.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="msg-6668386233557274254">
<div lang="EN-US" style="overflow-wrap: break-word;">
<div class="m_-6668386233557274254WordSection1">
<p class="MsoNormal">Hi,<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">I’m running MPICH 4.1.2.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Should a MPI user-defined datatype created by MPI_Type_create_struct have the same size as the C++ object on which it is based? Comparing sizeof(cpp_obj) and the output
<u></u><u></u></p>
<p class="MsoNormal">From MPI_Type_size(mpi_data_type_, &size1), the C++ object is four bytes larger than the MPI datatype. I don’t know if this is because I have to do something special to take
<u></u><u></u></p>
<p class="MsoNormal">padding into account when I create the MPI user datatype.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Here is my C++ message class:<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">struct MsgTupleSeq<u></u><u></u></p>
<p class="MsoNormal">{<u></u><u></u></p>
<p class="MsoNormal"> static const int N_INDICES_MAX_ = 4; // culprit?<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"> int n_vars_;<u></u><u></u></p>
<p class="MsoNormal"> int n_ndxs_;<u></u><u></u></p>
<p class="MsoNormal"> short start_[N_INDICES_MAX_];<u></u><u></u></p>
<p class="MsoNormal"> short end_[N_INDICES_MAX_];<u></u><u></u></p>
<p class="MsoNormal"> short current_[N_INDICES_MAX_];<u></u><u></u></p>
<p class="MsoNormal"> long long n_tuples_;<u></u><u></u></p>
<p class="MsoNormal"> int unique_id_;<u></u><u></u></p>
<p class="MsoNormal"> int subordinate_id_;<u></u><u></u></p>
<p class="MsoNormal"> int opcode_;<u></u><u></u></p>
<p class="MsoNormal"> char error_msg_[512]; <u></u><u></u></p>
<p class="MsoNormal">}<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">That is 564 bytes for the non-static members, but such an object is 568 bytes long with the 4 bytes of padding that the compiler inserts.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">I define the MPI user datatype with the code below, using the MsgTupleSeq object to compute the offsets. MPI_Type_size returns 564 bytes for the user datatype,
<u></u><u></u></p>
<p class="MsoNormal">so it seems like it isn’t seeing the padding.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"> MsgTupleSeq obj;<u></u><u></u></p>
<p class="MsoNormal"> MPI_Datatype mpi_data_type;<u></u><u></u></p>
<p class="MsoNormal"> int struct_len = 10, i = 0;<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"> int block_len[struct_len];<u></u><u></u></p>
<p class="MsoNormal"> MPI_Datatype types[struct_len];<u></u><u></u></p>
<p class="MsoNormal"> MPI_Aint displacements[struct_len];<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"> // the integer "n_vars_" member<u></u><u></u></p>
<p class="MsoNormal"> block_len[i] = 1;<u></u><u></u></p>
<p class="MsoNormal"> types[i] = MPI_INT;<u></u><u></u></p>
<p class="MsoNormal"> displacements[i] = (size_t) &obj.n_vars_ - (size_t) &obj;<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"> // the integer "n_ndxs_" member<u></u><u></u></p>
<p class="MsoNormal"> ++i;<u></u><u></u></p>
<p class="MsoNormal"> block_len[i] = 1;<u></u><u></u></p>
<p class="MsoNormal"> types[i] = MPI_INT;<u></u><u></u></p>
<p class="MsoNormal"> displacements[i] = (size_t) &obj.n_ndxs_ - (size_t) &obj;<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"> // the short integer "start_" array member<u></u><u></u></p>
<p class="MsoNormal"> ++i;<u></u><u></u></p>
<p class="MsoNormal"> block_len[i] = N_INDICES_MAX_;<u></u><u></u></p>
<p class="MsoNormal"> types[i] = MPI_SHORT;<u></u><u></u></p>
<p class="MsoNormal"> displacements[i] = (size_t) &obj.start_ - (size_t) &obj;<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"> // the short integer "end_" array member<u></u><u></u></p>
<p class="MsoNormal"> ++i;<u></u><u></u></p>
<p class="MsoNormal"> block_len[i] = N_INDICES_MAX_;<u></u><u></u></p>
<p class="MsoNormal"> types[i] = MPI_SHORT;<u></u><u></u></p>
<p class="MsoNormal"> displacements[i] = (size_t) &obj.end_ - (size_t) &obj;<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"> // the short integer "current_" array member<u></u><u></u></p>
<p class="MsoNormal"> ++i;<u></u><u></u></p>
<p class="MsoNormal"> block_len[i] = N_INDICES_MAX_;<u></u><u></u></p>
<p class="MsoNormal"> types[i] = MPI_SHORT;<u></u><u></u></p>
<p class="MsoNormal"> displacements[i] = (size_t) &obj.current_ - (size_t) &obj;<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"> // the long long integer "n_tuples_" member<u></u><u></u></p>
<p class="MsoNormal"> ++i;<u></u><u></u></p>
<p class="MsoNormal"> block_len[i] = 1;<u></u><u></u></p>
<p class="MsoNormal"> types[i] = MPI_LONG_LONG;<u></u><u></u></p>
<p class="MsoNormal"> displacements[i] = (size_t) &obj.n_tuples_ - (size_t) &obj;<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"> // the integer "unique_id_" member<u></u><u></u></p>
<p class="MsoNormal"> ++i;<u></u><u></u></p>
<p class="MsoNormal"> block_len[i] = 1;<u></u><u></u></p>
<p class="MsoNormal"> types[i] = MPI_INT;<u></u><u></u></p>
<p class="MsoNormal"> displacements[i] = (size_t) &obj.unique_id_ - (size_t) &obj;<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"> // the integer "subordinate_id_" member<u></u><u></u></p>
<p class="MsoNormal"> ++i;<u></u><u></u></p>
<p class="MsoNormal"> block_len[i] = 1;<u></u><u></u></p>
<p class="MsoNormal"> types[i] = MPI_INT;<u></u><u></u></p>
<p class="MsoNormal"> displacements[i] = (size_t) &obj.subordinate_id_ - (size_t) &obj;<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"> // the integer "opcode_" member<u></u><u></u></p>
<p class="MsoNormal"> ++i;<u></u><u></u></p>
<p class="MsoNormal"> block_len[i] = 1;<u></u><u></u></p>
<p class="MsoNormal"> types[i] = MPI_INT;<u></u><u></u></p>
<p class="MsoNormal"> displacements[i] = (size_t) &obj.opcode_ - (size_t) &obj;<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"> // the char array "error_msg_" member<u></u><u></u></p>
<p class="MsoNormal"> ++i;<u></u><u></u></p>
<p class="MsoNormal"> block_len[i] = 512;<u></u><u></u></p>
<p class="MsoNormal"> types[i] = MPI_CHAR;<u></u><u></u></p>
<p class="MsoNormal"> displacements[i] = (size_t) &obj.error_msg_[0] - (size_t) &obj;<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"> MPI_Type_create_struct(struct_len, block_len, displacements, types, &mpi_data_type);<u></u><u></u></p>
<p class="MsoNormal"> MPI_Type_commit(&mpi_data_type);<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
_______________________________________________<br>
discuss mailing list <a href="mailto:discuss@mpich.org" target="_blank">discuss@mpich.org</a><br>
To manage subscription options or unsubscribe:<br>
<a href="https://lists.mpich.org/mailman/listinfo/discuss" rel="noreferrer" target="_blank">https://lists.mpich.org/mailman/listinfo/discuss</a><br>
</div></blockquote></div>