Tell Me, What Do You See?—Interpretable Classification of Wiring Harness Branches with Deep Neural Networks

• Main Authors: Piotr Kicki, Michał Bednarek, Paweł Lembicz, Grzegorz Mierzwiak, Amadeusz Szymko, Marek Kraft, Krzysztof Walas
• Format: Article
• Language: English
• Published: MDPI AG 2021-06-01
• Series: Sensors
• Subjects: machine vision; deformable linear objects; neural networks; robot learning; computer vision for manufacturing
• Online Access: https://www.mdpi.com/1424-8220/21/13/4327

In the context of the robotisation of industrial operations related to manipulating deformable linear objects, there is a need for sophisticated machine vision systems, which could classify the wiring harness branches and provide information on where to put them in the assembly process. However, industrial applications require the interpretability of the machine learning system predictions, as the user wants to know the underlying reason for the decision made by the system. We propose several different neural network architectures that are tested on our novel dataset to address this issue. We conducted various experiments to assess the influence of modality, data fusion type, and the impact of data augmentation and pretraining. The outcome of the network is evaluated in terms of the performance and is also equipped with saliency maps, which allow the user to gain in-depth insight into the classifier’s operation, including a way of explaining the responses of the deep neural network and making system predictions interpretable by humans.