Communication Avoidance For Algorithms With Sparse All-To-All Interactions