Does SLOPE outperform bridge regression?

• Main Authors: Maleki, A. (Author), Wang, S. (Author), Weng, H. (Author)
• Format: Article
• Language: English
• Published: Oxford University Press 2022
• Subjects: 2000 Math Subject Classification; 34K30; 35K57; 35Q80; 92D25; Concentration inequality; LASSO; mean square error; noise sensitivity; Ridge; SLOPE
• Online Access: View Fulltext in Publisher

 A recently proposed SLOPE estimator [6] has been shown to adaptively achieve the minimax    ell _2   estimation rate under high-dimensional sparse linear regression models [25]. Such minimax optimality holds in the regime where the sparsity level     sample size    and dimension    satisfy    /p\rightarrow 0, k\log p/n\rightarrow 0    In this paper, we characterize the estimation error of SLOPE under the complementary regime where both    and    scale linearly with     and provide new insights into the performance of SLOPE estimators. We first derive a concentration inequality for the finite sample mean square error (MSE) of SLOPE. The quantity that MSE concentrates around takes a complicated and implicit form. With delicate analysis of the quantity, we prove that among all SLOPE estimators, LASSO is optimal for estimating    sparse parameter vectors that do not have tied nonzero components in the low noise scenario. On the other hand, in the large noise scenario, the family of SLOPE estimators are sub-optimal compared with bridge regression such as the Ridge estimator. © 2021 The Author(s) 2021. Published by Oxford University Press on behalf of the Institute of Mathematics and its Applications. All rights reserved.