Who counts? Survey data quality in the age of AI

von der Heyde, Leah

Vorschau

PDF
Dissertation_vonderHeyde_2025.pdf - Veröffentlichte Version
Download (12MB)

URN:	urn:nbn:de:bsz:180-madoc-709769
Dokumenttyp:	Dissertation
Erscheinungsjahr:	2025
Ort der Veröffentlichung:	Mannheim
Hochschule:	Universität Mannheim
Gutachter:	Prof. Dr. Florian Keusch
Datum der mündl. Prüfung:	2025
Sprache der Veröffentlichung:	Englisch
Einrichtung:	Fakultät für Sozialwissenschaften > Social Data Science and Methodology (Keusch 2022-)
Fachgebiet:	300 Sozialwissenschaften, Soziologie, Anthropologie
Freie Schlagwörter (Englisch):	survey methodology , public opinion , artificial intelligence , LLMs
Abstract:	Large language models (LLMs) have been hoped to make survey research more efficient, while also improving survey data quality. However, as they are based on Internet data, LLMs may come with similar potential pitfalls as other digital data sources with regard to making inferences about human attitudes and behavior. As such, they not only have the potential to mitigate, but also to amplify existing biases regarding our understanding of different populations and constructs of interest. In this dissertation, I investigate whether and under which conditions LLMs can be leveraged in survey research by providing empirical evidence of the potentials and limits of two major applications: supplementing survey data with LLM-generated data, and coding open-ended survey responses with LLMs. I test these applications in previously unexamined contexts – European countries and languages. I conclude that LLMs cannot fully replace, but could augment human-powered survey research, given proper supervision and validation.