mpi/pmix: Add developer pingpong test
Add some developer code that will allow to measure the latency of
the communication subsystem used.
Here is initial results for existing communication options:
Environment:
SLURM_PMIX_DIRECT_CONN=[true | false]
SLURM_PMIX_TIMEOUT=1000000
SLURM_PMIX_WANT_PP=1
SLURM_PMIX_PP_LOW_PWR2=0
SLURM_PMIX_PP_UP_PWR2=22
SLURM_PMIX_PP_ITER_SMALL=100
SLURM_PMIX_PP_ITER_LARGE=20
Launch cmdline:
$ srun --mpi=pmix -N2 -n2 sleep 10000
SLURM RPC results:
1 0.000669
2 0.000669
4 0.000669
8 0.000659
16 0.000668
32 0.000664
64 0.000668
128 0.000661
256 0.000668
512 0.000692
1024 0.000776
2048 0.000750
4096 0.000743
8192 0.000819
16384 0.001230
32768 0.001385
65536 0.002137
131072 0.003798
262144 0.006664
524288 0.011702
1048576 0.021605
2097152 0.042588
4194304 0.084673
Direct TCP connections
(better for large messages, but not yet for small):
1 0.078999
2 0.078999
4 0.078999
8 0.078999
16 0.078999
32 0.078999
64 0.079000
128 0.078999
256 0.079000
512 0.079000
1024 0.078998
2048 0.000409
4096 0.000639
8192 0.000521
16384 0.000618
32768 0.000933
65536 0.001622
131072 0.002749
262144 0.004704
524288 0.009281
1048576 0.018133
2097152 0.036172
4194304 0.071875
Signed-off-by:
Artem Polyakov <artpol84@gmail.com>
Please register or sign in to comment