An information theoretic approach for tracker performance evaluation

Automated tracking of vehicles and people is essential for the effective utilization of imagery in wide area surveillance applications. In order to determine the best tracking algorithm and parameters for a given application, a comprehensive evaluation procedure is required. However, despite half a...

Full description

Bibliographic Details
Main Authors: Kao, Edward K. (Contributor), Daggett, Matthew P. (Contributor), Hurley, Michael B. (Contributor)
Other Authors: Lincoln Laboratory (Contributor)
Format: Article
Language:English
Published: Institute of Electrical and Electronics Engineers, 2010-10-19T17:14:16Z.
Subjects:
Online Access:Get fulltext
LEADER 02554 am a22002533u 4500
001 59408
042 |a dc 
100 1 0 |a Kao, Edward K.  |e author 
100 1 0 |a Lincoln Laboratory  |e contributor 
100 1 0 |a Lincoln Laboratory  |e contributor 
100 1 0 |a Kao, Edward K.  |e contributor 
100 1 0 |a Kao, Edward K.  |e contributor 
100 1 0 |a Daggett, Matthew P.  |e contributor 
100 1 0 |a Hurley, Michael B.  |e contributor 
700 1 0 |a Daggett, Matthew P.  |e author 
700 1 0 |a Hurley, Michael B.  |e author 
245 0 0 |a An information theoretic approach for tracker performance evaluation 
260 |b Institute of Electrical and Electronics Engineers,   |c 2010-10-19T17:14:16Z. 
856 |z Get fulltext  |u http://hdl.handle.net/1721.1/59408 
520 |a Automated tracking of vehicles and people is essential for the effective utilization of imagery in wide area surveillance applications. In order to determine the best tracking algorithm and parameters for a given application, a comprehensive evaluation procedure is required. However, despite half a century of research in multi-target tracking, there is no consensus on how to score the overall performance of these trackers. Existing evaluation approaches assess tracker performance through measures of correspondence between ground truth tracks and system tracks using metrics such as track detection rate, track completeness, track fragmentation rate, and track ID change rate. However, each of these only provides a partial measure of performance and no good method exists to combine them into a holistic metric. Towards this end, this paper presents a pair of information theoretic metrics with similar behavior to the Receiver Operating Characteristic (ROC) curves of signal detection theory. Overall performance is evaluated with the percentage of truth information that a tracker captured and the total amount of false information that it reported. Information content is quantified through conditional entropy and mutual information computations using numerical estimates of the probability of association between the truth and the system tracks. This paper demonstrates how these information quality metrics provide a comprehensive evaluation of overall tracker performance and how they can be used to perform tracker comparisons and parameter tuning on wide-area surveillance imagery and other applications. 
520 |a United States. Dept. of Defense (Air Force Contract FA8721-05-C-0002) 
546 |a en_US 
655 7 |a Article 
773 |t 2009 IEEE 12th International Conference on Computer Vision