Audio-Visual Information Fusion for Biometric Verification System

• Main Authors: Yan-Ying Chen, 陳殷盈
• Other Authors: Shang-Hong Lai
• Format: Others
• Language: zh-TW
• Published: 2004
• Online Access: http://ndltd.ncl.edu.tw/handle/96602947378036041696

碩士 === 國立清華大學 === 資訊工程學系 === 92 === Personal verification system has already been used in real-life at present. It becomes increasingly important recently and has been employed in many practical applications, including automatic teller machines, access control system and so on. Biometrics is unique and can be classified between different claims. Hence, Biometrics can replace password or ID card to build a verification system. The multi-modal biometric verification system comprised of multiple experts would be more robust and more accurate than a single-modal biometrics verification system. The purpose of this thesis is to develop a reliable multi-modal biometric verification system based on speech and face information and to make this system more robust based on the features extracted from different experts. For multi-modal biometric verification systems, the features extracted from different modules need to be fused intelligently to improve the verification rate. So, we consider two fusion methods, i.e. opinion fusion and concatenation fusion, and two classifiers, i.e. Gaussian Mixture Models (GMMs) and Support Vector Machine (SVMs) in this thesis. Three audio-visual biometric verification systems formed by different fusion and classifiers are discussed in details. These three systems are compared with their verification rates on experiments on audio-visual biometric verification. In this thesis, we propose two SVM-based multi-modal biometric verification systems. The first system is based on the concatenation fusion with SVM classifier. This method concatenates the features provided by each expert to form a new feature vector. Then the SVM classifier is trained from the concatenated feature vectors for each person and later used for verification. To determine the final verification result from several SVM classification results for many possible paired audio-visual concatenated feature vectors, we present a new scheme for computing confidence weight based on the distance between feature vector and the hyper-plane of the associated SVM model. The final verification is determined from the weighted sum of all the SVM classification results. The idea for computing the confidence of the opinion can be used for SVM-based opinion fusion to enhance the verification rate. Finally, experimental results on the same audio-visual database for the three biometric verification systems are shown and their verification rates compared. We show that the proposed SVM-based fusion systems outperform the traditional GMM-based opinion fusion system.