<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hi Kurt,</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
MPI_Type_size returns the size of a datatype excluding the gaps (hole). You should use MPI_Type_get_extent to query the extent of the datatype, which should come out as 568.</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Alternatively, if you never going to break the struct apart during communication, you may try simply treating the struct as an opaque blob, i.e. use
<code>MPI_Type_contiguous(sizeof(struct MsgTupleSeq), MPI_BYTE, &newtype)</code>. It is likely to perform better by avoiding treating it as a non-contiguous datatype.</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
-- <br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hui<br>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Mccall, Kurt E. (MSFC-EV41) via discuss <discuss@mpich.org><br>
<b>Sent:</b> Sunday, December 3, 2023 2:55 PM<br>
<b>To:</b> discuss@mpich.org <discuss@mpich.org><br>
<b>Cc:</b> Mccall, Kurt E. (MSFC-EV41) <kurt.e.mccall@nasa.gov><br>
<b>Subject:</b> [mpich-discuss] MPI user-defined datatype size</font>
<div> </div>
</div>
<style>
<!--
@font-face
{font-family:"Cambria Math"}
@font-face
{font-family:Calibri}
p.x_MsoNormal, li.x_MsoNormal, div.x_MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif}
span.x_EmailStyle18
{font-family:"Calibri",sans-serif;
color:windowtext}
.x_MsoChpDefault
{font-size:10.0pt;
font-family:"Calibri",sans-serif}
@page WordSection1
{margin:1.0in 1.0in 1.0in 1.0in}
div.x_WordSection1
{}
-->
</style>
<div lang="EN-US" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="x_WordSection1">
<p class="x_MsoNormal">Hi,</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal">I’m running MPICH 4.1.2.</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal">Should a MPI user-defined datatype created by MPI_Type_create_struct have the same size as the C++ object on which it is based? Comparing sizeof(cpp_obj) and the output
</p>
<p class="x_MsoNormal">From MPI_Type_size(mpi_data_type_, &size1), the C++ object is four bytes larger than the MPI datatype. I don’t know if this is because I have to do something special to take
</p>
<p class="x_MsoNormal">padding into account when I create the MPI user datatype.</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal">Here is my C++ message class:</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal">struct MsgTupleSeq</p>
<p class="x_MsoNormal">{</p>
<p class="x_MsoNormal"> static const int N_INDICES_MAX_ = 4; // culprit?</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal"> int n_vars_;</p>
<p class="x_MsoNormal"> int n_ndxs_;</p>
<p class="x_MsoNormal"> short start_[N_INDICES_MAX_];</p>
<p class="x_MsoNormal"> short end_[N_INDICES_MAX_];</p>
<p class="x_MsoNormal"> short current_[N_INDICES_MAX_];</p>
<p class="x_MsoNormal"> long long n_tuples_;</p>
<p class="x_MsoNormal"> int unique_id_;</p>
<p class="x_MsoNormal"> int subordinate_id_;</p>
<p class="x_MsoNormal"> int opcode_;</p>
<p class="x_MsoNormal"> char error_msg_[512]; </p>
<p class="x_MsoNormal">}</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal">That is 564 bytes for the non-static members, but such an object is 568 bytes long with the 4 bytes of padding that the compiler inserts.</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal">I define the MPI user datatype with the code below, using the MsgTupleSeq object to compute the offsets. MPI_Type_size returns 564 bytes for the user datatype,
</p>
<p class="x_MsoNormal">so it seems like it isn’t seeing the padding.</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal"> MsgTupleSeq obj;</p>
<p class="x_MsoNormal"> MPI_Datatype mpi_data_type;</p>
<p class="x_MsoNormal"> int struct_len = 10, i = 0;</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal"> int block_len[struct_len];</p>
<p class="x_MsoNormal"> MPI_Datatype types[struct_len];</p>
<p class="x_MsoNormal"> MPI_Aint displacements[struct_len];</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal"> // the integer "n_vars_" member</p>
<p class="x_MsoNormal"> block_len[i] = 1;</p>
<p class="x_MsoNormal"> types[i] = MPI_INT;</p>
<p class="x_MsoNormal"> displacements[i] = (size_t) &obj.n_vars_ - (size_t) &obj;</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal"> // the integer "n_ndxs_" member</p>
<p class="x_MsoNormal"> ++i;</p>
<p class="x_MsoNormal"> block_len[i] = 1;</p>
<p class="x_MsoNormal"> types[i] = MPI_INT;</p>
<p class="x_MsoNormal"> displacements[i] = (size_t) &obj.n_ndxs_ - (size_t) &obj;</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal"> // the short integer "start_" array member</p>
<p class="x_MsoNormal"> ++i;</p>
<p class="x_MsoNormal"> block_len[i] = N_INDICES_MAX_;</p>
<p class="x_MsoNormal"> types[i] = MPI_SHORT;</p>
<p class="x_MsoNormal"> displacements[i] = (size_t) &obj.start_ - (size_t) &obj;</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal"> // the short integer "end_" array member</p>
<p class="x_MsoNormal"> ++i;</p>
<p class="x_MsoNormal"> block_len[i] = N_INDICES_MAX_;</p>
<p class="x_MsoNormal"> types[i] = MPI_SHORT;</p>
<p class="x_MsoNormal"> displacements[i] = (size_t) &obj.end_ - (size_t) &obj;</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal"> // the short integer "current_" array member</p>
<p class="x_MsoNormal"> ++i;</p>
<p class="x_MsoNormal"> block_len[i] = N_INDICES_MAX_;</p>
<p class="x_MsoNormal"> types[i] = MPI_SHORT;</p>
<p class="x_MsoNormal"> displacements[i] = (size_t) &obj.current_ - (size_t) &obj;</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal"> // the long long integer "n_tuples_" member</p>
<p class="x_MsoNormal"> ++i;</p>
<p class="x_MsoNormal"> block_len[i] = 1;</p>
<p class="x_MsoNormal"> types[i] = MPI_LONG_LONG;</p>
<p class="x_MsoNormal"> displacements[i] = (size_t) &obj.n_tuples_ - (size_t) &obj;</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal"> // the integer "unique_id_" member</p>
<p class="x_MsoNormal"> ++i;</p>
<p class="x_MsoNormal"> block_len[i] = 1;</p>
<p class="x_MsoNormal"> types[i] = MPI_INT;</p>
<p class="x_MsoNormal"> displacements[i] = (size_t) &obj.unique_id_ - (size_t) &obj;</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal"> // the integer "subordinate_id_" member</p>
<p class="x_MsoNormal"> ++i;</p>
<p class="x_MsoNormal"> block_len[i] = 1;</p>
<p class="x_MsoNormal"> types[i] = MPI_INT;</p>
<p class="x_MsoNormal"> displacements[i] = (size_t) &obj.subordinate_id_ - (size_t) &obj;</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal"> // the integer "opcode_" member</p>
<p class="x_MsoNormal"> ++i;</p>
<p class="x_MsoNormal"> block_len[i] = 1;</p>
<p class="x_MsoNormal"> types[i] = MPI_INT;</p>
<p class="x_MsoNormal"> displacements[i] = (size_t) &obj.opcode_ - (size_t) &obj;</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal"> // the char array "error_msg_" member</p>
<p class="x_MsoNormal"> ++i;</p>
<p class="x_MsoNormal"> block_len[i] = 512;</p>
<p class="x_MsoNormal"> types[i] = MPI_CHAR;</p>
<p class="x_MsoNormal"> displacements[i] = (size_t) &obj.error_msg_[0] - (size_t) &obj;</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal"> MPI_Type_create_struct(struct_len, block_len, displacements, types, &mpi_data_type);</p>
<p class="x_MsoNormal"> MPI_Type_commit(&mpi_data_type);</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal"> </p>
</div>
</div>
</body>
</html>