| Summary: | We develop a single joint model which can classify images and label super-pixels, based on tree-structured conditional random fields (CRFs) derived from a hierarchical image segmentation, extending previous work by Reynolds and Murphy, and Plath and Toussaint. We show how to train this model in a weakly-supervised fashion, in which some of the images only have captions specifying which objects are present; this information is propagated down the tree and thus provides weakly labeled data at the leaves, which can be used to improve the performance of the super-pixel classifiers. After training, information can be propagated from the super-pixels up to the root-level image classifier (although this does not seem to help in practice compared to just using root-level features). We compare two kinds of tree: the standard one with pairwise potentials, and one based on noisy-or potentials, which better matches the semantics of the recursive partitioning used to create the tree. However, we do not find any significant difference between the two. === Science, Faculty of === Computer Science, Department of === Graduate
|
|---|
