Enhancing theory-informed dictionary approaches with “glass-box” machine learning: The case of integrative complexity in social media comments

Dobbrick, Timo ; Jakob, Julia ; Chan, Chung-hong ; Wessler, Hartmut

DOI: https://doi.org/10.1080/19312458.2021.1999913
URL: https://www.tandfonline.com/doi/full/10.1080/19312...
Additional URL: https://www.researchgate.net/publication/356339614...
Document Type: Article
Year of publication: 2021
The title of a journal, publication series: Communication Methods and Measures
Volume: 16
Issue number: 4
Page range: 303-320
Place of publication: Philadelphia, PA
Publishing house: Routledge, Taylor & Francis Group
ISSN: 1931-2458 , 1931-2466
Publication language: English
Institution: Außerfakultäre Einrichtungen > Mannheim Centre for European Social Research - Research Department B
School of Humanities > Medien- und Kommunikationswissenschaft (Wessler 2007-)
Subject: 070 News media, journalism, publishing
Abstract: Dictionary-based approaches to computational text analysis have been shown to perform relatively poorly, particularly when the dictionaries rely on simple bags of words, are not specified for the domain under study, and add word scores without weighting. While machine learning approaches usually perform better, they offer little insight into (a) which of the assumptions underlying dictionary approaches (bag-of-words, domain transferability, or additivity) impedes performance most, and (b) which language features drive the algorithmic classification most strongly. To fill both gaps, we offer a systematic assumption-based error analysis, using the integrative complexity of social media comments as our case in point. We show that attacking the additivity assumption offers the strongest potential for improving dictionary performance. We also propose to combine off-the-shelf dictionaries with supervised “glass box” machine learning algorithms (as opposed to the usual “black box” machine learning approaches) to classify texts and learn about the most important features for classification. This dictionary-plus-supervised-learning approach performs similarly well as classic full-text machine learning or deep learning approaches, but yields interpretable results in addition, which can inform theory development on top of enabling a valid classification.

Dieser Eintrag ist Teil der Universitätsbibliographie.

Metadata export


+ Search Authors in

+ Page Views

Hits per month over past year

Detailed information

You have found an error? Please let us know about your desired correction here: E-Mail

Actions (login required)

Show item Show item