On The Effectiveness of Multi-TaskLearningAn evaluation of Multi-Task Learning techniques in deep learning models

• Main Author: Tovedal, Sofiea
• Format: Others
• Language: English
• Published: Umeå universitet, Institutionen för datavetenskap 2020
• Subjects: Multi-Task Learning; Natural Language Processing; Supervised Learning; Gated Recurrent Unit; Long Short Term Memory; Engineering and Technology; Teknik och teknologier
• Online Access: http://urn.kb.se/resolve?urn=urn:nbn:se:umu:diva-172257

Multi-Task Learning is today an interesting and promising field which many mention as a must for achieving the next level advancement within machine learning. However, in reality, Multi-Task Learning is much more rarely used in real-world implementations than its more popular cousin Transfer Learning. The questionis why that is and if Multi-Task Learning outperforms its Single-Task counterparts. In this thesis different Multi-Task Learning architectures were utilized in order to build a model that can handle labeling real technical issues within two categories. The model faces a challenging imbalanced data set with many labels to choose from and short texts to base its predictions on. Can task-sharing be the answer to these problems? This thesis investigated three Multi-Task Learning architectures and compared their performance to a Single-Task model. An authentic data set and two labeling tasks was used in training the models with the method of supervised learning. The four model architectures; Single-Task, Multi-Task, Cross-Stitched and the Shared-Private, first went through a hyper parameter tuning process using one of the two layer options LSTM and GRU. They were then boosted by auxiliary tasks and finally evaluated against each other.