Entropy under additive Bernoulli and spherical noises

• Main Authors: Ordentlich, Or (Author), Polyanskiy, Yury (Author)
• Other Authors: Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science (Contributor)
• Format: Article
• Language: English
• Published: Institute of Electrical and Electronics Engineers (IEEE), 2020-05-01T20:42:19Z.
• Subjects: Article
• Online Access: Get fulltext

 Let Z[superscript n] be iid Bernoulli (δ) and U[superscript n] be uniform on the set of all binary vectors of weight δ[superscript n] (Hamming sphere). As is well known, the entropies of Z[superscript n] and U[superscript n] are within O(√n). However, if X[superscript n] is another binary random variable independent of Z[superscript n] and U[superscript n], we show that H(X[superscript n]+U[superscript n]) and H(X[superscript n]+Z[superscript n]) are within O(√n) and this estimate is tight. The bound is shown via coupling method. Tightness follows from the observation that the channels x[superscript n]⟼x[superscript n]+U[superscript n] and x[superscript n]⟼x[superscript n]+Z[superscript n] have similar capacities, but the former has zero dispersion. Finally, we show that despite the √n slack in general, the Mrs. Gerber Lemma for H(X[superscript n]+U[superscript n]) holds with only an O(log n) correction compared to its brethren for H(X[superscript n]+Z[superscript n]). ©2019 Paper presented at the 2018 IEEE International Symposium on Information Theory (ISIT 2018), June 17-22, 2018, Vail, Colo.