Optimization Algorithms on Riemannian Manifolds with Applications

• Other Authors: Huang, Wen (authoraut)
• Format: Others
• Language: English; English
• Published: Florida State University
• Subjects: Mathematics
• Online Access: http://purl.flvc.org/fsu/fd/FSU_migr_etd-8809

This dissertation generalizes three well-known unconstrained optimization approaches for Rn to solve optimization problems with constraints that can be viewed as a d-dimensional Riemannian manifold to obtain the Riemannian Broyden family of methods, the Riemannian symmetric rank-one trust region method, and Riemannian gradient sampling method. The generalization relies on basic differential geometric concepts, such as tangent spaces, Riemannian metrics, and the Riemannian gradient, as well as on the more recent notions of (first-order) retraction and vector transport. The effectiveness of the methods and techniques for their efficient implementation are derived and evaluated. Basic experiments and applications are used to illustrate the value of the proposed methods. Both the Riemannian symmetric rank-one trust region method and the RBroyden family of methods are generalized from Euclidean quasi-Newton optimization methods, in which a Hessian approximation exploits the well-known secant condition. The generalization of the secant condition and the associated update formulas that define quasi-Newton methods to the Riemannian setting is a key result of this dissertation. The dissertation also contains convergence theory for these methods. The Riemannian symmetric rank-one trust region method is shown to converge globally to a stationary point and d+1-step q-superlinearly to a minimizer of the objective function. The RBroyden family of methods is shown to converge globally and q-superlinearly to a minimizer of a retraction-convex objective function. A condition, called the locking condition, on vector transport and retraction that guarantees convergence for the RBroyden family method and facilitates efficient computation is derived and analyzed. The Dennis Mor\'e sufficient and necessary conditions for superlinear convergence, can be generalized to the Riemannian setting in multiple ways. This dissertation generalizes them in a novel manner that is applicable to both Riemannian optimization problems and root finding for a vector field on a Riemannian manifold. The convergence analyses of Riemannian symmetric rank-one trust region method and RBroyden family methods assume a smooth objective function. For partly smooth Lipschitz continuous objective functions, a variation of one of the RBroyden family methods, RBFGS, is shown to be work well empirically. In addition, the Riemannian gradient sampling method is shown to work well empirically for both a Lipschitz continuous and a non-Lipschitz continuous objective function associated with the important application nonlinear dimension reduction. Efficient and effective implementations for a manifold in Rn, a quotient manifold of total manifold in Rn and a product of manifolds, are presented. Results include efficient representations and operations of elements in a manifold, tangent vectors, linear operators, retractions and vector transports. Novel techniques for constructing and computing multiple kinds of vector transports are derived. In addition, the implementation details of all required objects for optimization on four manifolds, the Stiefel manifold, the sphere, the orthogonal group and the Grassmann manifold, are presented. Basic numerical experiments for the Brockett cost function on the Stiefel manifold, the Rayleigh quotient on the Grassmann manifold and the minmax problem on the sphere (Lipschitz and non-Lipschitz forms), are used to illustrate the performance of the proposed methods and compare with existing optimization methods on manifolds. Three applications, Riemannian optimization for elastic shape analysis, a joint diagonalization problem for independent component analysis and a synchronization of rotation problem, that have smooth cost functions are used to show the advantages of the proposed methods. A secant-based nonlinear dimension reduction problem with a partly smooth function is used to show the advantages of the Riemannian gradient sampling method. === A Dissertation submitted to the Department of Mathematics in partial fulfillment of the requirements for the degree of Doctor of Philosophy. === Fall Semester, 2013. === November 5, 2013. === Optimization, quasi-Newton, Reparameterization, Retraction and Vector Transport, Riemannian manifold, Stiefel and Grassmann === Includes bibliographical references. === Kyle A. Gallivan, Professor Directing Dissertation; Pierre-Antoine Absil, Professor Directing Dissertation; Dennis Duke, University Representative; Giray Okten, Committee Member; Eric P. Klassen, Committee Member.