The online Gradient Balancing (GraB) algorithm, which determines example orderings through the herding problem with per-sample gradients, has been proved as a theoretically optimal solution, consistently outperforming Random Reshuffling. Yet, a user-friendly and efficient implementation is absent in the community.

This work presents GraB-sampler, an efficient and easy-to-use Python library designed for seamless integration of the GraB algorithm, additionally proposing three novel variants. The best performance result of the GraB-sampler reproduces the training loss and test accuracy results with only the cost of 8.7% training time overhead and 0.85% peak GPU memory usage overhead.

De Novo Genome assembly is one of the most important tasks in computational biology. ELBA is the state-of-the-art distributed-memory parallel algorithm for overlap detection and layout simplification steps of De Novo genome assembly but exists a performance bottleneck in pairwise alignment.

In this work, we introduce 3 GPU schedulers for ELBA to accommodate multiple MPI processes and multiple GPUs. The GPU schedulers enable multiple MPI processes to perform computation on GPUs in a round-robin fashion. Both strong and weak scaling experiments show that 3 schedulers are able to significantly improve the performance of baseline while there is a trade-off between parallelism and GPU scheduler overhead. For the best performance implementation, the one-to-one scheduler achieves ~7-8x speed-up using 25 MPI processes compared with the baseline vanilla ELBA GPU scheduler.