Submitted

SHANG++: Robust Stochastic Acceleration under Multiplicative Noise

Yaxin Yu, Long Chen, and Minfu Feng

Submitted

arXiv   Bibtex

ABSTRACT:

Under the multiplicative noise scaling (MNS) condition,
original Nesterov acceleration is provably sensitive to noise and may
diverge when gradient noise overwhelms the signal. In this paper, we
develop two accelerated stochastic gradient descent methods by
discretizing the Hessian-driven Nesterov accelerated gradient flow. We
first derive SHANG, a direct semi-implicit discretization that already
improves stability under MNS. We then introduce SHANG++, which adds a
damping correction and achieves faster convergence with greater noise
robustness. We establish convergence guarantees for both convex and
strongly convex objectives under MNS, together with explicit parameter
choices. In our experiments, SHANG++ performs consistently well across
convex problems and applications in deep learning. In a dedicated
noise experiment on ResNet-34, a single hyperparameter configuration
maintains accuracy within one percentage point of the noise-free
setting. Across all experiments, SHANG++ outperforms existing
accelerated methods in robustness and efficiency, with minimal
parameter sensitivity.