INVESTIGATIONS ON THE POTENTIAL OF CONVOLUTIONAL NEURAL NETWORKS FOR VEHICLE CLASSIFICATION BASED ON RGB AND LIDAR DATA

• Main Authors: R. Niessner, H. Schilling, B. Jutzi
• Format: Article
• Language: English
• Published: Copernicus Publications 2017-05-01
• Series: ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences
• Online Access: http://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/IV-1-W1/115/2017/isprs-annals-IV-1-W1-115-2017.pdf

In recent years, there has been a significant improvement in the detection, identification and classification of objects and images using Convolutional Neural Networks. To study the potential of the Convolutional Neural Network, in this paper three approaches are investigated to train classifiers based on Convolutional Neural Networks. These approaches allow Convolutional Neural Networks to be trained on datasets containing only a few hundred training samples, which results in a successful classification. Two of these approaches are based on the concept of transfer learning. In the first approach features, created by a pretrained Convolutional Neural Network, are used for a classification using a support vector machine. In the second approach a pretrained Convolutional Neural Network gets fine-tuned on a different data set. The third approach includes the design and training for flat Convolutional Neural Networks from the scratch. The evaluation of the proposed approaches is based on a data set provided by the IEEE Geoscience and Remote Sensing Society (GRSS) which contains RGB and LiDAR data of an urban area. In this work it is shown that these Convolutional Neural Networks lead to classification results with high accuracy both on RGB and LiDAR data. Features which are derived by RGB data transferred into LiDAR data by transfer learning lead to better results in classification in contrast to RGB data. Using a neural network which contains fewer layers than common neural networks leads to the best classification results. In this framework, it can furthermore be shown that the practical application of LiDAR images results in a better data basis for classification of vehicles than the use of RGB images.