Bayesian Learning of Latent Representations of Language Structures

• Main Author: Yugo Murawaki
• Format: Article
• Language: English
• Published: The MIT Press 2019-06-01
• Series: Computational Linguistics
• Online Access: https://www.mitpressjournals.org/doi/pdf/10.1162/coli_a_00346

We borrow the concept of representation learning from deep learning research, and we argue that the quest for Greenbergian implicational universals can be reformulated as the learning of good latent representations of languages, or sequences of surface typological features. By projecting languages into latent representations and performing inference in the latent space, we can handle complex dependencies among features in an implicit manner. The most challenging problem in turning the idea into a concrete computational model is the alarmingly large number of missing values in existing typological databases. To address this problem, we keep the number of model parameters relatively small to avoid overfitting, adopt the Bayesian learning framework for its robustness, and exploit phylogenetically and/or spatially related languages as additional clues. Experiments show that the proposed model recovers missing values more accurately than others and that some latent variables exhibit phylogenetic and spatial signals comparable to those of surface features.