Evaluation of English–Slovak Neural and Statistical Machine Translation

• Main Authors: Lucia Benkova, Dasa Munkova, Ľubomír Benko, Michal Munk
• Format: Article
• Language: English
• Published: MDPI AG 2021-03-01
• Series: Applied Sciences
• Subjects: neural machine translation; statistical machine translation; text analysis; automatic evaluation; Slovak language; English language
• Online Access: https://www.mdpi.com/2076-3417/11/7/2948

This study is focused on the comparison of phrase-based statistical machine translation (SMT) systems and neural machine translation (NMT) systems using automatic metrics for translation quality evaluation for the language pair of English and Slovak. As the statistical approach is the predecessor of neural machine translation, it was assumed that the neural network approach would generate results with a better quality. An experiment was performed using residuals to compare the scores of automatic metrics of the accuracy (BLEU_n) of the statistical machine translation with those of the neural machine translation. The results showed that the assumption of better neural machine translation quality regardless of the system used was confirmed. There were statistically significant differences between the SMT and NMT in favor of the NMT based on all BLEU_n scores. The neural machine translation achieved a better quality of translation of journalistic texts from English into Slovak, regardless of if it was a system trained on general texts, such as Google Translate, or specific ones, such as the European Commission’s (EC’s) tool, which was trained on a specific-domain.