Inspired by recent work on the quantile-randomized Kaczmarz method (qRK) for solving a linear system of equations, we propose an acceleration of the randomized Kaczmarz method using quantile information. We show that the method converges faster than the randomized Kaczmarz algorithm when the linear system is consistent. In addition, we demonstrate how this new acceleration may be used in conjunction with qRK, without additional computational complexity, to produce both a fast and robust iterative method for solving large, sparsely corrupted linear systems. Finally, we provide new error horizon bounds for qRK in the setting where the corruption may not be sparse.