SensePOLAR: Word sense aware interpretability for pre-trained contextual word embeddings

Engler, Jan ; Sikdar, Sandipan ; Lutz, Marlene ; Strohmaier, Markus

Document Type: Conference or workshop publication
Year of publication: 2023
Book title: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing : EMNLP 2022
Page range: 4607-4619
Conference title: EMNLP 2022
Location of the conference venue: Abu Dhabi, United Arab Emirates
Date of the conference: 07.-11.12.2022
Publisher: Goldberg, Yoav ; Kozareva, Zornitsa ; Zhang, Yue
Place of publication: Stroudsburg, PA
Publishing house: ACL
Related URLs:
Publication language: English
Institution: Business School > Data Science in the Economic and Social Sciences (Strohmaier, 2022-)
Subject: 400 Language, linguistics
Abstract: Adding interpretability to word embeddings represents an area of active research in text representation. Recent work has explored thepotential of embedding words via so-called polar dimensions (e.g. good vs. bad, correct vs. wrong). Examples of such recent approaches include SemAxis, POLAR, FrameAxis, and BiImp. Although these approaches provide interpretable dimensions for words, they have not been designed to deal with polysemy, i.e. they can not easily distinguish between different senses of words. To address this limitation, we present SensePOLAR, an extension of the original POLAR framework that enables word-sense aware interpretability for pre-trained contextual word embeddings. The resulting interpretable word embeddings achieve a level of performance that is comparable to original contextual word embeddings across a variety of natural language processing tasks including the GLUE and SQuAD benchmarks. Our work removes a fundamental limitation of existing approaches by offering users sense aware interpretations for contextual word embeddings.

Dieser Eintrag ist Teil der Universitätsbibliographie.

Metadata export


+ Search Authors in

+ Page Views

Hits per month over past year

Detailed information

You have found an error? Please let us know about your desired correction here: E-Mail

Actions (login required)

Show item Show item