Optimization with block variables: theory and applications.

• Other Authors: Chen, Bilian
• Format: Others
• Language: English; Chinese
• Published: 2012
• Subjects: Mathematical optimization; Nonlinear theories > Mathematical models; Engineering mathematics
• Online Access: http://library.cuhk.edu.hk/record=b5549571; http://repository.lib.cuhk.edu.hk/en/item/cuhk-328412

本博士论文对于有结构但又有相当一般性的约束条件下的非线性优化问题给出了系统性研究。比较经典的例子包指球面约束下的多重线性函数优化问题。这些模型已被广泛应用于数值线性代数、材料科学、量子物理学、信号处理、语音识别、生物医学工程以及控制论等。本论文着重探讨一类特定的方法来解这些广义模型，即块变量改进方法。具体地说，我们构造了一类块坐标下降型搜索算法来解带块变量结构的非线性优化问题。这类算法通过每次迭代中只更新一块变量以达到最大限度的目标函数值的改进(因而，这一新搜索算法命名为最优块改进算法(简称为MBI)。之后，我们重点研究了该算法在求解众多领域中实际问题的潜在能力。首先，这一算法可以直接应用于生物信息学中聚类基因表达数据的一种新模型的设计及求解。接着，我们把注意力转移到球面约束下的齐次多项式优化问题，此问题与张量的最优秩-1 逼近问题相关。对于这一优化问题， MBI 算法通常可以在较少的计算时间内找到全局最优解。第三，我们继续深入研究多项式优化问题，在双协半正定的新概念下建立了齐次多项式优化问题与其多重线性优化问题关系的一般性结果。最后，我们在Tucker 分解的框架下给出了求解高阶张量的最优多重线性秩的逼近问题的方法，并提出一种新的模型和算法来解未知变量数的Tucker 分解问题。本论文讨论并试验了一些应用实例，数值实验表明所提出的算法分别对于求解以上这些问题是可行并有效的。 === In this thesis we present a systematic analysis for optimization of a general nonlinear function, subject to some fairly general constraints. A typical example includes the optimization of a multilinear tensor function over spherical constraints. Such models have found wide applications in numerical linear algebra, material sciences, quantum physics, signal processing, speech recognition, biomedical engineering, and control theory. This thesis is mainly concerned with a specific approach to solve such generic models: the block variable improvement method. Specifically, we establish a block coordinate descent type search method for nonlinear optimization, which accepts only a block update that achieves the maximum improvement (hence the name of our new search method: maximum block improvement (MBI)). Then, we focus on the potential capability of this method for solving problems in various research area. First, we demonstrate that this method can be directly used in designing a new framework for co-clustering gene expression data in the area of bioinformatics. Second, we turn our attention to the spherically constrained homogeneous polynomial optimization problem, which is related to best rank-one approximation of tensors. The MBI method usually finds the global optimal solution at a low computational cost. Third, we continue to consider polynomial optimization problems. A general result between homogeneous polynomials and multi-linear forms under the concept of co-quadratic positive semidefinite is established. Finally, we consider the problem of finding the best multi-linear rank approximation of a higher-order tensor under the framework of Tucker decomposition, and also propose a new model and algorithms for computing Tucker decomposition with unknown number of components. Some real application examples are discussed and tested, and numerical experiments are reported to reveal the good practical performance and efficiency of the proposed algorithms for solving those problems. === Detailed summary in vernacular field only. === Chen, Bilian. === Thesis (Ph.D.)--Chinese University of Hong Kong, 2012. === Includes bibliographical references (leaves 86-98). === Abstract also in Chinese. === Abstract --- p.iii === Acknowledgements --- p.vii === Chapter 1 --- Introduction --- p.1 === Chapter 1.1 --- Overview --- p.1 === Chapter 1.2 --- Notations and Preliminaries --- p.5 === Chapter 1.2.1 --- The Tensor Operations --- p.6 === Chapter 1.2.2 --- The Tensor Ranks --- p.9 === Chapter 1.2.3 --- Polynomial Functions --- p.11 === Chapter 2 --- The Maximum Block Improvement Method --- p.12 === Chapter 2.1 --- Introduction --- p.12 === Chapter 2.2 --- MBI Method and Convergence Analysis --- p.14 === Chapter 3 --- Co-Clustering of Gene Expression Data --- p.19 === Chapter 3.1 --- Introduction --- p.19 === Chapter 3.2 --- A New Generic Framework for Co-Clustering Gene Expression Data --- p.22 === Chapter 3.2.1 --- Tensor Optimization Model of The Co-Clustering Problem --- p.22 === Chapter 3.2.2 --- The MBI Method for Co-Clustering Problem --- p.23 === Chapter 3.3 --- Algorithm for Co-Clustering 2D Matrix Data --- p.25 === Chapter 3.4 --- Numerical Experiments --- p.27 === Chapter 3.4.1 --- Implementation Details and Some Discussions --- p.27 === Chapter 3.4.2 --- Testing Results using Microarray Datasets --- p.30 === Chapter 3.4.3 --- Testing Results using 3D Synthesis Dataset --- p.32 === Chapter 4 --- Polynomial Optimization with Spherical Constraint --- p.34 === Chapter 4.1 --- Introduction --- p.34 === Chapter 4.2 --- Generalized Equivalence Result --- p.37 === Chapter 4.3 --- Spherically Constrained Homogeneous Polynomial Optimization --- p.41 === Chapter 4.3.1 --- Implementing MBI on Multilinear Tensor Form --- p.42 === Chapter 4.3.2 --- Relationship between Homogeneous Polynomial Optimization over Spherical Constraint and Tensor Relaxation Form --- p.43 === Chapter 4.3.3 --- Finding a KKT point for Homogeneous Polynomial Optimization over Spherical Constraint --- p.45 === Chapter 4.4 --- Numerical Experiments on Randomly Simulated Data --- p.47 === Chapter 4.4.1 --- Multilinear Tensor Function over Spherical Constraints --- p.49 === Chapter 4.4.2 --- Tests of Another Implementation of MBI --- p.49 === Chapter 4.4.3 --- General Polynomial Function over Quadratic Constraints --- p.51 === Chapter 4.5 --- Applications --- p.53 === Chapter 4.5.1 --- Rank-One Approximation of Super-Symmetric Tensors --- p.54 === Chapter 4.5.2 --- Magnetic Resonance Imaging --- p.55 === Chapter 5 --- Logarithmically Quasiconvex Optimization --- p.58 === Chapter 5.1 --- Introduction --- p.58 === Chapter 5.2 --- Logarithmically Quasiconvex Optimization --- p.60 === Chapter 5.2.1 --- A Simple Motivating Example --- p.61 === Chapter 5.2.2 --- Co-Quadratic Positive Semide nite Tensor Form --- p.61 === Chapter 5.2.3 --- Equivalence at Maxima --- p.64 === Chapter 6 --- The Tucker Decomposition and Generalization --- p.68 === Chapter 6.1 --- Introduction --- p.68 === Chapter 6.2 --- Convergence of Traditional Tucker Decomposition --- p.71 === Chapter 6.3 --- Tucker Decomposition with Unknown Number of Components --- p.73 === Chapter 6.3.1 --- Problem Formulation --- p.74 === Chapter 6.3.2 --- Implementing the MBI Method on Tucker Decomposition with Unknown Number of Components --- p.75 === Chapter 6.3.3 --- A Heuristic Approach --- p.79 === Chapter 6.4 --- Numerical Experiments --- p.80 === Chapter 7 --- Conclusion and Recent Developments --- p.83 === Bibliography --- p.86