Multidimensional adaption of language models: domains, languages, and social dimensions

Hung, Chia-Chien

PDF
Dissertation_Hung (submit).pdf - Veröffentlichte Version
Download (3MB)

URN:	urn:nbn:de:bsz:180-madoc-689942
Dokumenttyp:	Dissertation
Erscheinungsjahr:	2024
Ort der Veröffentlichung:	Mannheim
Hochschule:	Universität Mannheim
Gutachter:	Ponzetto, Simone Paolo
Sprache der Veröffentlichung:	Englisch
Einrichtung:	Fakultät für Wirtschaftsinformatik und Wirtschaftsmathematik > Information Systems III: Enterprise Data Analysis (Ponzetto 2016-)
Fachgebiet:	004 Informatik
Freie Schlagwörter (Englisch):	pre-trained language model , adaptation barrier , domain adaptation , language adaptation , demographic adaptation , multidimensional adaptation , transfer learning , cross-lingual transfer , task-agnostic adaptation
Abstract:	The emergence of Pre-trained Language Models (PLMs) has revolutionized natural language processing, yielding remarkable improvements across diverse tasks. While PLMs excel at discerning language patterns and crafting coherent narratives, they face challenges when adapting to specialized topics, such as medical concepts, cross-lingual conversational systems, and content from diverse sociodemographic backgrounds. These challenges, collectively referred to as the "adaptation barrier," stem from the difficulty of tailoring PLMs to fields requiring expert knowledge, often due to insufficient topic-relevant information within the model’s training data. Recent research has explored techniques like domain adaptation, cross-lingual transfer learning, and the integration of external knowledge to address these limitations. However, existing approaches often focus narrowly on either domain-specific, language-specific, or social dimensions, limiting a comprehensive understanding of the effectiveness of proposed adaptation approaches. In this thesis, we systematically investigate the adaptation barrier by conducting experiments across multiple dimensions. We focus on three key challenges: effectiveness, efficiency, and interpretability in adapting PLMs to diverse domains, languages, and social dimensions. Through our findings, we hope to contribute to the development of more adaptable language models across multiple dimensions, enhancing their applicability in addressing real-world challenges, thereby mitigating the adaptation barrier inherent to PLMs.