When Will a Sequence of Points in a Riemannian Submanifold Converge?

• Main Author: Tuyen Trung Truong
• Format: Article
• Language: English
• Published: MDPI AG 2020-11-01
• Series: Mathematics
• Subjects: compact metric space; deep neural networks; random dynamical systems; global convergence of Gradient Descent; iterative optimisation; Nash conjecture
• Online Access: https://www.mdpi.com/2227-7390/8/11/1934

Let X be a Riemannian manifold and <inline-formula><math display="inline"><semantics><msub><mi>x</mi><mi>n</mi></msub></semantics></math></inline-formula> a sequence of points in X. Assume that we know a priori some properties of the set A of cluster points of <inline-formula><math display="inline"><semantics><msub><mi>x</mi><mi>n</mi></msub></semantics></math></inline-formula>. The question is under what conditions that <inline-formula><math display="inline"><semantics><msub><mi>x</mi><mi>n</mi></msub></semantics></math></inline-formula> will converge. An answer to this question serves to understand the convergence behaviour for iterative algorithms for (constrained) optimisation problems, with many applications such as in Deep Learning. We will explore this question, and show by some examples that having X a submanifold (more generally, a metric subspace) of a good Riemannian manifold (even in infinite dimensions) can greatly help.