Capsule Network Improved Multi-Head Attention for Word Sense Disambiguation

• Main Authors: Jinfeng Cheng, Weiqin Tong, Weian Yan
• Format: Article
• Language: English
• Published: MDPI AG 2021-03-01
• Series: Applied Sciences
• Subjects: word sense disambiguation; multi-head attention; capsule network; capsule routing
• Online Access: https://www.mdpi.com/2076-3417/11/6/2488

Word sense disambiguation (WSD) is one of the core problems in natural language processing (NLP), which is to map an ambiguous word to its correct meaning in a specific context. There has been a lively interest in incorporating sense definition (gloss) into neural networks in recent studies, which makes great contribution to improving the performance of WSD. However, disambiguating polysemes of rare senses is still hard. In this paper, while taking gloss into consideration, we further improve the performance of the WSD system from the perspective of semantic representation. We encode the context and sense glosses of the target polysemy independently using encoders with the same structure. To obtain a better presentation in each encoder, we leverage the capsule network to capture different important information contained in multi-head attention. We finally choose the gloss representation closest to the context representation of the target word as its correct sense. We do experiments on English all-words WSD task. Experimental results show that our method achieves good performance, especially having an inspiring effect on disambiguating words of rare senses.