Modify backfill algorithm
Modify backfill algorithm to improve performance with large numbers of running jobs. Group running jobs that end in a "similar" time frame using a time window that grows exponentially rather than linearly. The original window sizes were (in units of minutes): 0, 1, 2, 3, 4, 5, 6, 7, ... minutes The new window sizes are: 0.5, 1, 2, 4, 8, 16, 32, ... minutes This can dramatically reduce the number of instances where the very time consuming "can the pending job run now" operation is executed, especailly if there are 1000+ running jobs. bug 3275
Please register or sign in to comment