Message quantization in belief propagation: Structural results in the low-rate regime

• Main Authors: Willsky, Alan S. (Contributor), Kreidl, Olivier Patrick (Contributor)
• Other Authors: Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science (Contributor)
• Format: Article
• Language: English
• Published: Institute of Electrical and Electronics Engineers, 2010-10-22T18:16:05Z.
• Subjects: Article
• Online Access: Get fulltext

Motivated by distributed inference applications in unreliable communication networks, we adapt the popular (sum-product) belief propagation (BP) algorithm under the constraint of discrete-valued messages. We show that, in contrast to conventional BP, the optimal message-generation rules are node-dependent and iteration-dependent, each rule making explicit use of local memory from all past iterations. These results expose both the intractability of optimal design and an inherent structure that can be exploited for tractable approximate design. We propose one such approximation and demonstrate its efficacy on canonical examples. We also discuss extensions to communication networks with lossy links (e.g., erasures) or topologies that differ from the graph underlying the probabilistic model.
United States. Army Research Office (ARO MURI W911NF-06-1-0076)
United States. Air Force Office of Scientific Research (MURI FA9550-06-1-0324)