Learning to Optimize Under Non-Stationarity

• Main Authors: Cheung, Wang Chi (Author), Simchi-Levi, David (Author), Zhu, Ruihao (Author)
• Format: Article
• Language: English
• Published: Elsevier BV, 2021-11-02T12:19:41Z.
• Subjects: Article
• Online Access: Get fulltext

 © 2019 by the author(s). We introduce algorithms that achieve state-of-the-art dynamic regret bounds for non-stationary linear stochastic bandit setting. It captures natural applications such as dynamic pricing and ads allocation in a changing environment. We show how the difficulty posed by the non-stationarity can be overcome by a novel marriage between stochastic and adversarial bandits learning algorithms. Defining d, BT, and T as the problem dimension, the variation budget, and the total time horizon, respectively, our main contributions are the tuned Sliding Window UCB (SW-UCB) algorithm with optimal Oe(d2/3(BT + 1)1/3T2/3) dynamic regret, and the tuning free bandit-over-bandit (BOB) framework built on top of the SW-UCB algorithm with best Oe(d2/3(BT + 1)1/4T3/4) dynamic regret.