Tracing on the right side of the brain: unsupervised image simplification and vectorization

• Main Author: Olsen, Sven Crandall
• Other Authors: Gooch, Bruce
• Language: English; en
• Published: 2010
• Subjects: Computer Graphics; Image Processing; UVic Subject Index::Sciences and Engineering::Applied Sciences::Computer science
• Online Access: http://hdl.handle.net/1828/3017

I present an unsupervised system which takes digital photographs as input, and generates simplified, stylized vector data as output. The three component parts of the system are image-space stylization, edge tracing, and edge-based image reconstruction. The design of each of these components is specialized, relative to their state of the art equivalents, in order to improve their effectiveness when used in such a combined stylization / vectorization pipeline. I demonstrate that the vector data generated by this system is often both an effective visual simplification of the input photographs, and an effective simplification in the sense of memory efficiency, as judged relative to state of the art lossy image compression formats. Many recent image-based stylization algorithms are designed to simplify or abstract the contents of a source image; creating cartoon-like results. An ideal cartoon simplification preserves the important semantics of the image, while de-emphasizing unimportant visual details. However, the benefits of any such visual simplifications are limited because the output of an image-based stylization is another image. In order to fully exploit image simplification in a software engineering context, an abstracted image must be "simpler" not just in terms of its apparent visual complexity, but also in terms of the number of bits needed to represent it. At present, the most robust image abstraction algorithms produce results that are merely visually simpler than their source data; the storage requirements of the "simplified" result is often exactly that of the initial image. In contrast to computationally stylized images are vector-graphic cartoons, created by a human artist from a reference image. Vector art is more easily modified than bitmap images, and it can be a more memory efficient image representation. However, the only reliable way to generate vector cartoons from source images is to employ a human artist, and thus the advantages of vector art cannot be exploited in fully automatic systems. In this work, I approach image-based stylization, edge tracing, and edge-based image reconstruction with the assumption that the three tasks are synergistic. I describe an unsupervised system which takes digital photographs as input and uses them to create stylized vector art, resulting in a simplification of the source data in terms of bit encoding costs, as well as visual complexity. The specific algorithms that comprise this system are modified relative to the current state of the art in order to take better advantage of the complementary nature of the component tasks. My primary technical contributions are: 1) I show that the edge modeling problem, previously identified as one of the fundamental challenges facing edge-only image representations, has a relatively simple and robust solution, in the special case of images that have been stylized using aggressive smoothing followed by soft quantization. 2) I introduce a novel edge-based image reconstruction method, which differs from prior work in that anisotropic regularization is used in place of a varying width Gaussian blur. While previous vector formats have successfully used varying width blurring to model soft edges, the technique leads to artifacts given the unusually large widths required by the traced vector data. My anisotropic regularization approach avoids these artifacts, while maintaining a high degree of reconstruction accuracy. 3) I demonstrate that the vector data generated by my system is, in the sense of memory efficiency, significantly simpler than the input photographs. Specifically, we compare our vector output with state of the art lossy image compression results. While my vector encodings are in no sense accurate reproductions of the input photographs, they do maintain a sharp, stylized look, while preserving most visually important elements. The results of general purpose compression codecs suffer from significant visual artifacts at similar file sizes.