Robust Shelf Monitoring Using Supervised Learning for Improving On-Shelf Availability in Retail Stores

• Main Authors: Kyota Higa, Kota Iwamoto
• Format: Article
• Language: English
• Published: MDPI AG 2019-06-01
• Series: Sensors
• Subjects: image processing; change detection; change classification; background subtraction; convolutional neural network; on-shelf availability; product amount; surveillance camera; retail
• Online Access: https://www.mdpi.com/1424-8220/19/12/2722

This paper proposes a method to robustly monitor shelves in retail stores using supervised learning for improving on-shelf availability. To ensure high on-shelf availability, which is a key factor for improving profits in retail stores, we focus on understanding changes in products regarding increases/decreases in product amounts on the shelves. Our method first detects changed regions of products in an image by using background subtraction followed by moving object removal. It then classifies the detected change regions into several classes representing the actual changes on the shelves, such as “product taken (decrease)” and “product replenished/returned (increase)”, by supervised learning using convolutional neural networks. It finally updates the shelf condition representing the presence/absence of products using classification results and computes the product amount visible in the image as on-shelf availability using the updated shelf condition. Three experiments were conducted using two videos captured from a surveillance camera on the ceiling in a real store. Results of the first and second experiments show the effectiveness of the product change classification in our method. Results of the third experiment show that our method achieves a success rate of 89.6% for on-shelf availability when an error margin is within one product. With high accuracy, store clerks can maintain high on-shelf availability, enabling retail stores to increase profits.