<div dir="ltr">Hi Min,<div><br></div><div>If that is the case, please ignore this email. Nothing is wrong without OrangeFS direct interface. I will try "ch4:ucx". Thank you for the info.</div><div><br clear="all"><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">Thanks<div>Kun</div></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Oct 6, 2019 at 10:25 AM Si, Min via discuss <<a href="mailto:discuss@mpich.org">discuss@mpich.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
Hi Kun,<br>
<br>
Can you please try to reproduce the issue in a simple MPI program which does not use OrangeFS ? It is hard for the MPICH community to help when mixing MPI and OrangeFS together, because we are not OrangeFS experts.<br>
<br>
Besides, for InfiniBand networks, you might want to use `ch4:ucx` instead of `ch4:ofi`. But I do not think it causes the failure in your use case.<br>
<br>
Best regards,<br>
Min<br>
<br>
<div>On 2019/10/04 12:21, Kun Feng via discuss wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">To Whom It May Concern,
<div><br>
</div>
<div>Recently, I switched to CH4 device in MPICH 3.3.1 for better network performance over the RoCE network we are using.</div>
<div>I realized that my code fails to run when I use direct interface of OrangeFS 2.9.7. It exits without any error. But even simple helloworld cannot print anything. It happens only when I enable direct interface of OrangeFS by linking -lorangefsposix.</div>
<div>Could you please help me on this issue?</div>
<div>Here are some information that might be useful:</div>
<div>Output of ibv_devinfo of 40Gbps Mellanox ConnectX-4 Lx adapter:</div>
<div>hca_id: mlx5_0<br>
transport: InfiniBand (0)<br>
fw_ver: 14.20.1030<br>
node_guid: 248a:0703:0015:a800<br>
sys_image_guid: 248a:0703:0015:a800<br>
vendor_id: 0x02c9<br>
vendor_part_id: 4117<br>
hw_ver: 0x0<br>
board_id: LNV2430110027<br>
phys_port_cnt: 1<br>
port: 1<br>
state: PORT_ACTIVE (4)<br>
max_mtu: 4096 (5)<br>
active_mtu: 1024 (3)<br>
sm_lid: 0<br>
port_lid: 0<br>
port_lmc: 0x00<br>
link_layer: Ethernet<br>
<br>
hca_id: i40iw0<br>
transport: iWARP (1)<br>
fw_ver: 0.2<br>
node_guid: 7cd3:0aef:3da0:0000<br>
sys_image_guid: 7cd3:0aef:3da0:0000<br>
vendor_id: 0x8086<br>
vendor_part_id: 14289<br>
hw_ver: 0x0<br>
board_id: I40IW Board ID<br>
phys_port_cnt: 1<br>
port: 1<br>
state: PORT_ACTIVE (4)<br>
max_mtu: 4096 (5)<br>
active_mtu: 1024 (3)<br>
sm_lid: 0<br>
port_lid: 1<br>
port_lmc: 0x00<br>
link_layer: Ethernet<br>
</div>
<div>MPICH 3.3.1 configuration command: ./configure --with-device=ch4:ofi --with-pvfs2=/home/kfeng/install --enable-shared --enable-romio --with-file-system=ufs+pvfs2+zoidfs --enable-fortran=no --with-libfabric=/home/kfeng/install</div>
<div>OrangeFS 2.9.7 configuration command: ./configure --prefix=/home/kfeng/install --enable-shared --enable-jni --with-jdk=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-0.el7_5.x86_64 --with-kernel=/usr/src/kernels/3.10.0-862.el7.x86_64</div>
<div>Make command: mpicc -o ~/hello ~/hello.c -L/home/kfeng/install/lib -lorangefsposix</div>
<div>The verbose outputs of mpiexec are attached.</div>
<div><br clear="all">
<div>
<div dir="ltr">
<div dir="ltr">Thanks
<div>Kun</div>
</div>
</div>
</div>
</div>
</div>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
discuss mailing list <a href="mailto:discuss@mpich.org" target="_blank">discuss@mpich.org</a>
To manage subscription options or unsubscribe:
<a href="https://lists.mpich.org/mailman/listinfo/discuss" target="_blank">https://lists.mpich.org/mailman/listinfo/discuss</a>
</pre>
</blockquote>
<br>
</div>
_______________________________________________<br>
discuss mailing list <a href="mailto:discuss@mpich.org" target="_blank">discuss@mpich.org</a><br>
To manage subscription options or unsubscribe:<br>
<a href="https://lists.mpich.org/mailman/listinfo/discuss" rel="noreferrer" target="_blank">https://lists.mpich.org/mailman/listinfo/discuss</a><br>
</blockquote></div>