Adaptive learning in lasso models

• Main Author: Patnaik, Kaushik
• Other Authors: Song, Le
• Format: Others
• Language: en_US
• Published: Georgia Institute of Technology 2016
• Subjects: Lasso; L1 regression; Adaptive methods; Active learning
• Online Access: http://hdl.handle.net/1853/54353

Regression with L1-regularization, Lasso, is a popular algorithm for recovering the sparsity pattern (also known as model selection) in linear models from observations contaminated by noise. We examine a scenario where a fraction of the zero co-variates are highly correlated with non-zero co-variates making sparsity recovery difficult. We propose two methods that adaptively increment the regularization parameter to prune the Lasso solution set. We prove that the algorithms achieve consistent model selection with high probability while using fewer samples than traditional Lasso. The algorithm can be extended to a broad set of L1-regularized M-estimators for linear statistical models.