Submitted

Accelerated Gradient Methods Through Variable and Operator Splitting

Long Chen, Hao Luo, and Jingrong Wei

Submitted

arXiv   Bibtex

ABSTRACT:

This paper introduces a unified framework for accelerated
gradient methods through the variable and operator splitting
(VOS). The operator splitting decouples the optimization process into
simpler subproblems, and more importantly, the variable splitting
leads to acceleration. The key contributions include the development
of strong Lyapunov functions to analyze stability and convergence
rates, as well as advanced discretization techniques like Accelerated
Over-Relaxation (AOR) and extrapolation by the predictor-corrector
methods (EPC). For convex case, we introduce a dynamic updating
parameter and a perturbed VOS flow. The framework effectively handles
a wide range of optimization problems, including convex optimization,
composite convex optimization, and saddle point systems with bilinear
coupling.