Function Analysis of the Euclidean Distance between Probability Distributions

• Main Author: Namyong Kim
• Format: Article
• Language: English
• Published: MDPI AG 2018-01-01
• Series: Entropy
• Subjects: distribution; recursive; gradient; spreading; functions
• Online Access: http://www.mdpi.com/1099-4300/20/1/48

Minimization of the Euclidean distance between output distribution and Dirac delta functions as a performance criterion is known to match the distribution of system output with delta functions. In the analysis of the algorithm developed based on that criterion and recursive gradient estimation, it is revealed in this paper that the minimization process of the cost function has two gradients with different functions; one that forces spreading of output samples and the other one that compels output samples to move close to symbol points. For investigation the two functions, each gradient is controlled separately through individual normalization of each gradient with their related input. From the analysis and experimental results, it is verified that one gradient is associated with the role of accelerating initial convergence speed by spreading output samples and the other gradient is related with lowering the minimum mean squared error (MSE) by pulling error samples close together.