Minimax Rates of <i>ℓ</i>_<i>p</i>-Losses for High-Dimensional Linear Errors-in-Variables Models over <i>ℓ</i>_<i>q</i>-Balls

• Main Authors: Xin Li, Dongya Wu
• Format: Article
• Language: English
• Published: MDPI AG 2021-06-01
• Series: Entropy
• Subjects: sparse linear regression; errors-in-variables model; minimax rate; Kullback–Leibler divergence; information-theoretic limitations
• Online Access: https://www.mdpi.com/1099-4300/23/6/722

In this paper, the high-dimensional linear regression model is considered, where the covariates are measured with additive noise. Different from most of the other methods, which are based on the assumption that the true covariates are fully obtained, results in this paper only require that the corrupted covariate matrix is observed. Then, by the application of information theory, the minimax rates of convergence for estimation are investigated in terms of the <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><msub><mo>ℓ</mo><mi>p</mi></msub><mspace width="4pt"></mspace><mrow><mo>(</mo><mn>1</mn><mo>≤</mo><mi>p</mi><mo><</mo><mo>∞</mo><mo>)</mo></mrow></mrow></semantics></math></inline-formula>-losses under the general sparsity assumption on the underlying regression parameter and some regularity conditions on the observed covariate matrix. The established lower and upper bounds on minimax risks agree up to constant factors when <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mi>p</mi><mo>=</mo><mn>2</mn></mrow></semantics></math></inline-formula>, which together provide the information-theoretic limits of estimating a sparse vector in the high-dimensional linear errors-in-variables model. An estimator for the underlying parameter is also proposed and shown to be minimax optimal in the <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msub><mo>ℓ</mo><mn>2</mn></msub></semantics></math></inline-formula>-loss.