Evaluating representation bias of different survey recruitment designs

Rohr, Björn

Vorschau

PDF
Dissertation_Rohr_2025_public.pdf - Veröffentlichte Version
Download (13MB)

URN:	urn:nbn:de:bsz:180-madoc-711931
Dokumenttyp:	Dissertation
Erscheinungsjahr:	2025
Ort der Veröffentlichung:	Mannheim
Hochschule:	Universität Mannheim
Gutachter:	Keusch, Florian
Datum der mündl. Prüfung:	2025
Sprache der Veröffentlichung:	Englisch
Einrichtung:	Fakultät für Sozialwissenschaften > Social Data Science and Methodology (Keusch 2022-)
Fachgebiet:	300 Sozialwissenschaften, Soziologie, Anthropologie
Freie Schlagwörter (Deutsch):	Auswahlverzerrung , Zufallsstichprobe , Nicht-Zufallsstichprobe , Facebook-Umfragen , Piggybacking-Rekrutierung , Verzerrung bei der Schätzung von Korrelationen
Freie Schlagwörter (Englisch):	selection bias , probability surveys , nonprobability surveys , facebook surveys , piggybacking recruitment , bias in estimates of correlations
Abstract:	Most studies in the social sciences strongly rely on accurate and unbiased survey estimates. Whether you use probability or nonprobability sampling, each survey method comes with its own advantages and disadvantages, but will they yield accurate univariate, bivariate, or multivariate estimates? In this dissertation, I address this question by comparing surveys collected through various methods with external benchmarks. The first study compares probability and nonprobability surveys against population benchmarks to evaluate whether univariate, bivariate, and multivariate estimates yield similarly accurate results. In the second study, I compare non-probability surveys, recruited with advertisements on Facebook, with two different methods to target members of the target population. While the first method – Simple Demographic Targeting – is less complicated to conduct, the alternative method – Complex Demographic Targeting – was often used in the past, as it is suggested that this method might mitigate the bias introduced by the Facebook Ads Distribution Algorithm. Nonetheless, this suggestion was rarely evaluated in the past. In a third study, I analyze whether piggybacking, a cost-efficient method of conducting a probability survey, where the survey respondents are recruited at the end of another survey, leads to higher amounts of nonresponse bias in univariate and bivariate estimates. Finally, I provide practical guidance on the circumstances under which each of the compared methods is more or less suitable, and I outline avenues for further methodological development in the field.