[mpich-discuss] MPICH (v3.1) Performing slow on 10G Ethernet

[Muhammad Ansar Javed]

Hi,
I am running benchmarks for MPICH v3.1 performance evaluation on 10G
Ethernet connection between two hosts. The performance results are less
than expected. Here is complete set of numbers for Latency and Bandwidth
tests.

mpj at host3:~/code/benchmarks$ mpiexec -n 2 -f machines ./latency.out
Latency: MPICH-C
Size ______ Time (us)
1 _________ 37.91
2 _________ 37.76
4 _________ 37.75
8 _________ 39.02
16 ________ 52.71
32 ________ 39.08
64 ________ 37.75
128 _______ 37.77
256 _______ 57.47
512 _______ 37.86
1024 ______ 37.76
2048 ______ 37.88

mpj at host3:~/code/benchmarks$ mpiexec -n 2 -f machines ./bandwidth.out
Bandwidth: MPICH-C
Size(Bytes) Bandwidth (Mbps)
2048 ______ 412.32
4096 ______ 820.06
8192 ______ 827.77
16384 _____ 1644.36
32768 _____ 2207.52
65536 _____ 4368.76
131072 ____ 2942.93
262144 ____ 4281.17
524288 ____ 4773.78
1048576 ___ 5310.85
2097152 ___ 5382.94
4194304 ___ 5518.97
8388608 ___ 5508.87
16777216 __ 5498.93

My environments consists of two hosts having point-to-point (switch-less)
10Gbps Ethernet connection.  Environment (OS, user, directory structure
etc) on both hosts is exactly same. There is no NAS or shared file system
between hosts. I have attached output of script
mpiexec -all.

Are these numbers okay? If not then please suggest performance improvement
methodology...

Thanks

-- 
Regards

Ansar Javed
HPC Lab
SEECS NUST
Contact: +92 334 438 9394
Email: muhammad.ansar at seecs.edu.pk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20140415/ef741c35/attachment.html>
-------------- next part --------------
HYDRA build details:
    Version:                                 3.1
    Release Date:                            Thu Feb 20 11:41:13 CST 2014
    CC:                              gcc   
    CXX:                             no   
    F77:                             no  
    F90:                             no  
    Configure options:                       '--disable-option-checking' '--prefix=/home/mpj/installed/mpich3-install' '--disable-cxx' '--disable-f77' '--disable-fc' '--cache-file=/dev/null' '--srcdir=.' 'CC=gcc' 'CFLAGS= -O2' 'LDFLAGS= ' 'LIBS=-lrt -lpthread ' 'CPPFLAGS= -I/home/mpj/software/mpich-3.1/src/mpl/include -I/home/mpj/software/mpich-3.1/src/mpl/include -I/home/mpj/software/mpich-3.1/src/openpa/src -I/home/mpj/software/mpich-3.1/src/openpa/src -I/home/mpj/software/mpich-3.1/src/mpi/romio/include'
    Process Manager:                         pmi
    Launchers available:                     ssh rsh fork slurm ll lsf sge manual persist
    Topology libraries available:            hwloc
    Resource management kernels available:   user slurm ll lsf sge pbs cobalt
    Checkpointing libraries available:      
    Demux engines available:                 poll select