<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Segoe UI";
panose-1:2 11 5 2 4 2 4 2 2 3;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
code
{mso-style-priority:99;
font-family:"Courier New";}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
font-size:10.0pt;
font-family:"Courier New";}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:"Consolas",serif;}
span.EmailStyle22
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Hi Rupsa,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Could you try the latest release, mpich-4.0, for your application? This waiting for server finalization behavior may have been fixed.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<div>
<p class="MsoNormal">-- <br>
Hui Zhou<o:p></o:p></p>
</div>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:12.0pt;margin-left:.5in">
<b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">Rupsa Chakraborty via discuss <discuss@mpich.org><br>
<b>Date: </b>Thursday, January 27, 2022 at 1:42 PM<br>
<b>To: </b>discuss@mpich.org <discuss@mpich.org><br>
<b>Cc: </b>Rupsa Chakraborty <c.rupsa@gmail.com><br>
<b>Subject: </b>[mpich-discuss] MPICH Usage Issue: Client process not exiting till server reaches MPI_Finalize() in MPI client server singleton INIT connection<o:p></o:p></span></p>
</div>
<div>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:5.0pt;margin-left:.5in;vertical-align:baseline;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;line-height:inherit;box-sizing:inherit">
<span style="font-size:11.5pt;font-family:"Segoe UI",sans-serif;color:#232629">Hello,<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:5.0pt;margin-left:.5in;vertical-align:baseline;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;line-height:inherit;box-sizing:inherit">
<span style="font-size:11.5pt;font-family:"Segoe UI",sans-serif;color:#232629">I am asking this question in this forum as I had already asked this question in stack overflow and did not receive any answer/comment.<br>
<br>
I am trying to implement a feature that performs a large computation in parallel over multiple machines. I am trying out zeroMQ as well as MPI for this, and will accept the one which gives better performance and also is better in ease of usage and support.
Your help in this is much appreciated.<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:5.0pt;margin-left:.5in;vertical-align:baseline;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;line-height:inherit;box-sizing:inherit">
<span style="font-size:11.5pt;font-family:"Segoe UI",sans-serif;color:#232629">I am implementing a new feature, using MPI client server parallel computation with 'Singleton INIT' mechanism, in an existing large application on Linux. I am using mpich-3.4.1 for
this. The main process becomes an mpi server on certain user inputs. Client processes connect to this server and share a large parallel computation between themselves and return the result back to the server. The clients and server processes are independently
started (Singleton INIT). All client processes should be able to normally exit after disconnecting from the server, even though the server is still running (doing some other work, but not accepting any more client connections). <o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:5.0pt;margin-left:.5in;vertical-align:baseline;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;line-height:inherit;box-sizing:inherit">
<span style="font-size:11.5pt;font-family:"Segoe UI",sans-serif;color:#232629">My questions is:<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:5.0pt;margin-left:.5in;vertical-align:baseline;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;line-height:inherit;box-sizing:inherit">
<span style="font-size:11.5pt;font-family:"Segoe UI",sans-serif;color:#232629">Is it possible to keep the server process running and executing other tasks, while all of its client connections have closed and client processes exited?<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:5.0pt;margin-left:.5in;vertical-align:baseline;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;line-height:inherit;box-sizing:inherit">
<span style="font-size:11.5pt;font-family:"Segoe UI",sans-serif;color:#232629">In my client-server MPI code, I see that for the last client connection to a server (corresponding to the last MPI_Comm_accept() call) the client process always gets stuck at its
MPI_Finalize() till the server reaches its own MPI_Finalize(). Thus the last client process does not exit till the server reaches its MPI_Finalize() stage. However, for the previous client connections the client processes are able to normally exit.<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:5.0pt;margin-left:.5in;vertical-align:baseline;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;line-height:inherit;box-sizing:inherit">
<span style="font-size:11.5pt;font-family:"Segoe UI",sans-serif;color:#232629">'lsof' command shows, at the server side, call to MPI_Comm_accept() releases the file descriptor from the previous connection, which is not getting released for the last connection
due to the absence of subsequent MPI_Comm_accept() call, but gets released at the MPI_Finalize() call.<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:5.0pt;margin-left:.5in;vertical-align:baseline;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;line-height:inherit;box-sizing:inherit">
<span style="font-size:11.5pt;font-family:"Segoe UI",sans-serif;color:#232629">What is it that I am missing or doing incorrectly? How to have the last client also exit and not wait at MPI_Finalize() stage in the following code, even though the server is still
running?<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:5.0pt;margin-left:.5in;vertical-align:baseline;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;line-height:inherit;box-sizing:inherit">
<span style="font-size:11.5pt;font-family:"Segoe UI",sans-serif;color:#232629">Is there any way I can set a timeout to the MPI_Comm_accept? or interrupt it from a separate thread?<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:12.0pt;margin-left:.5in;vertical-align:baseline;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;line-height:inherit;box-sizing:inherit">
<span style="font-size:11.5pt;font-family:"Segoe UI",sans-serif;color:#232629">My client and server code are as follows:<o:p></o:p></span></p>
<pre style="margin-left:.5in;vertical-align:baseline;white-space:pre-wrap;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;box-sizing:inherit;max-height:600px;border-radius:5px;overflow:auto"><code><span style="border:none windowtext 1.0pt;padding:0in">//server.cxx<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">int main() {<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> char ch;<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> MPI_Init(NULL, NULL);<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> char portName[MPI_MAX_PORT_NAME]; <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> MPI_Open_port(MPI_INFO_NULL, portName); <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> publishServerPortNameToFile(portName, /*args*/); <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> MPI_Comm intercomm1; <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> // First connection. Client corresponding to this connection connects and exits successfully. <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> MPI_Comm_accept(portName, MPI_INFO_NULL, 0, MPI_COMM_SELF, &intercomm1); <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> std::cout << "\nConnection1 Accepted.";<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> MPI_Comm intercomm2; <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> // Second connection. Client corresponding to this connection connects, but waits at its <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> // own MPI_Finalize() stage, and exits only after server reaches MPI_finalize(). <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> MPI_Comm_accept(portName, MPI_INFO_NULL, 0, MPI_COMM_SELF, &intercomm2); <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> std::cout << "\nConnection 2 Accepted.";<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> std::cout << "\nBefore disconnect."; <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> MPI_Close_port(portName); <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> MPI_Comm_disconnect(&intercomm1); <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> MPI_Comm_disconnect(&intercomm2); <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> std::cout << "\nBefore Finalize()."; <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> MPI_Finalize(); <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> std::cout << "\nAfter Finalize."; <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">}<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"><o:p> </o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">//client.cxx<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">int main() {<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> MPI_Init(NULL, NULL);<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> std::string portName = getServerPortNameFromFile(/*args*/);<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"><o:p> </o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> MPI_Comm intercomm; <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> MPI_Comm_connect(portName.c_str(), MPI_INFO_NULL, 0, MPI_COMM_SELF, &intercomm);<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> std::cout << "\nConnected"; <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> <o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> MPI_Comm_disconnect(&intercomm);<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"><o:p> </o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> std::cout << "\nBefore Finalize()";<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> MPI_Finalize();<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"> std::cout << "\nAfter Finalize().\nClient Exited";<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">}</span></code><o:p></o:p></pre>
<pre style="margin-left:.5in;vertical-align:baseline;white-space:pre-wrap;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;box-sizing:inherit;max-height:600px;border-radius:5px;overflow:auto"><o:p> </o:p></pre>
<pre style="margin-left:.5in;vertical-align:baseline;white-space:pre-wrap;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;box-sizing:inherit;max-height:600px;border-radius:5px;overflow:auto"><code><span style="border:none windowtext 1.0pt;padding:0in"><o:p> </o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"><o:p> </o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline;white-space:pre-wrap;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;box-sizing:inherit;max-height:600px;border-radius:5px;overflow:auto"><code><span style="border:none windowtext 1.0pt;padding:0in">Output after first client connection is made, and the client exits:<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"><o:p> </o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">shell1:<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">$ ./server<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">Connection 1 Accepted<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">< server waiting to accept next connection ><o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"><o:p> </o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">shell2:<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">$ ./client // first client requesting connection<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">Connected<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">Before Finalize()<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">After Finalize()<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">Client Exited<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">$<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"><o:p> </o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">Output after the second client requests connection (at this point the first client had successfully connected and exited):<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"><o:p> </o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">shell1:<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">$ ./server<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">Connection 1 Accepted<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">Connection 2 Accepted.<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">Before disconnect. Press any key // key pressed<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">Before Finalize(). Press any key // key not yet pressed<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"><o:p> </o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">shell2:<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">$ ./client // this is the second client connection to the same server<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in"><o:p> </o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">Connected<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">Before Finalize().<o:p></o:p></span></code></pre>
<pre style="margin-left:.5in;vertical-align:baseline"><code><span style="border:none windowtext 1.0pt;padding:0in">< this client process waits here till the server reaches its Finalize() ></span></code><span style="border:none windowtext 1.0pt;padding:0in"><o:p></o:p></span></pre>
<pre style="margin-left:.5in;vertical-align:baseline;white-space:pre-wrap;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;box-sizing:inherit;max-height:600px;border-radius:5px;overflow:auto"><span style="border:none windowtext 1.0pt;padding:0in"><o:p> </o:p></span></pre>
<pre style="margin-left:.5in;vertical-align:baseline;white-space:pre-wrap;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;box-sizing:inherit;max-height:600px;border-radius:5px;overflow:auto"><span style="border:none windowtext 1.0pt;padding:0in"><o:p> </o:p></span></pre>
<pre style="margin-left:.5in;vertical-align:baseline;white-space:pre-wrap;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;box-sizing:inherit;max-height:600px;border-radius:5px;overflow:auto"><code><span style="border:none windowtext 1.0pt;padding:0in">Regards,</span></code><span style="border:none windowtext 1.0pt;padding:0in"><o:p></o:p></span></pre>
<pre style="margin-left:.5in;vertical-align:baseline;white-space:pre-wrap;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;box-sizing:inherit;max-height:600px;border-radius:5px;overflow:auto"><code><span style="border:none windowtext 1.0pt;padding:0in">Rupsa</span></code><span style="border:none windowtext 1.0pt;padding:0in"><o:p></o:p></span></pre>
</div>
</div>
</body>
</html>