Enhancing theory-informed dictionary approaches with “glass-box” machine learning: The case of integrative complexity in social media comments

Dobbrick, Timo ; Jakob, Julia ; Chan, Chung-hong ; Wessler, Hartmut

Preview

PDF
Enhancing Theory-Informed Dictionary Approaches with Glass-box Machine Learning The Case of Integrative Complexity in Social Media Comments.pdf - Published
Download (1MB)

DOI:	https://doi.org/10.1080/19312458.2021.1999913
URL:	https://www.tandfonline.com/doi/full/10.1080/19312...
Additional URL:	https://www.researchgate.net/publication/356339614...
URN:	urn:nbn:de:bsz:180-madoc-609497
Document Type:	Article
Year of publication:	2021
The title of a journal, publication series:	Communication Methods and Measures
Volume:	16
Issue number:	4
Page range:	303-320
Place of publication:	Philadelphia, PA
Publishing house:	Routledge, Taylor & Francis Group
ISSN:	1931-2458 , 1931-2466
Publication language:	English
Institution:	Außerfakultäre Einrichtungen > Mannheim Centre for European Social Research - Research Department B School of Humanities > Medien- und Kommunikationswissenschaft (Wessler 2007-)
Pre-existing license:	Creative Commons Attribution, Non-Commercial, No Derivatives 4.0 International (CC BY-NC-ND 4.0)
Subject:	070 News media, journalism, publishing
Abstract:	Dictionary-based approaches to computational text analysis have been shown to perform relatively poorly, particularly when the dictionaries rely on simple bags of words, are not specified for the domain under study, and add word scores without weighting. While machine learning approaches usually perform better, they offer little insight into (a) which of the assumptions underlying dictionary approaches (bag-of-words, domain transferability, or additivity) impedes performance most, and (b) which language features drive the algorithmic classification most strongly. To fill both gaps, we offer a systematic assumption-based error analysis, using the integrative complexity of social media comments as our case in point. We show that attacking the additivity assumption offers the strongest potential for improving dictionary performance. We also propose to combine off-the-shelf dictionaries with supervised “glass box” machine learning algorithms (as opposed to the usual “black box” machine learning approaches) to classify texts and learn about the most important features for classification. This dictionary-plus-supervised-learning approach performs similarly well as classic full-text machine learning or deep learning approaches, but yields interpretable results in addition, which can inform theory development on top of enabling a valid classification.

	Dieser Eintrag ist Teil der Universitätsbibliographie.
	Das Dokument wird vom Publikationsserver der Universitätsbibliothek Mannheim bereitgestellt.

Search Authors in

BASE: Dobbrick, Timo ; Jakob, Julia ; Chan, Chung-hong ; Wessler, Hartmut

Google Scholar: Dobbrick, Timo ; Jakob, Julia ; Chan, Chung-hong ; Wessler, Hartmut

ORCID: Dobbrick, Timo