• Rémi Palancher's avatar
    avoid delay on commit for PMI task at rank 0 · fcc11e22
    Rémi Palancher authored
    Intel MPI, on MPI jobs initialisation through PMI, uses to call PMI_KVS_Put()
    many many times from task at rank 0, and each on these call is followed by
    PMI_KVS_Commit(). Slurm implementation of PMI_KVS_Commit() imposes a delay
    to avoid DDOS on original srun. This delay is proportional to the total number.
    It could be up to 3 secs for large jobs for ex. with 7168 tasks. Therefore,
    when Intel MPI calls PMI_KVS_Commit() 475 times (mesured on a test case) from
    task at rank 0, 28 minutes are spent in delay function.
    All other tasks in the job are waiting for a PMI_Barrier. Therefore, there is
    no risk for a DDOS from this single task 0. The patch alters the delaying time
    calculation to make sure task at rank 0 will does not be delayed. All other
    tasks are globally spreaded in the same time range as before.
    fcc11e22
To find the state of this project's repository at the time of any of these versions, check out the tags.