WDC products: A multi-dimensional entity matching benchmark

Peeters, Ralph ; Der, Reng Chiz ; Bizer, Christian

PDF
paper-14.pdf - Published
Download (761kB)

DOI:	https://doi.org/10.48786/edbt.2024.03
URN:	urn:nbn:de:bsz:180-madoc-651334
Document Type:	Conference or workshop publication
Year of publication:	2023
Book title:	Proceedings 27th International Conference on Extending Database Technology (EDBT 2024), Paestum, Italy, March 25 - March 28
The title of a journal, publication series:	OpenProceedings
Volume:	27
Page range:	22-33
Conference title:	International Conference on Extending Database Technology (EDBT 2024)
Location of the conference venue:	Paestum, Italy
Date of the conference:	25.-28.05.2024
Publisher:	Tanca, Letizia ; Luo, Qiong ; Polese, Giuseppe ; Caruccio, Loredana ; Oriol, Xavier ; Firmani, Donatella
Place of publication:	Konstanz
Publishing house:	OpenProceedings.org
ISBN:	978-3-89318-091-2
ISSN:	2367-2005
Related URLs:	https://dastlab.github.io/edbticdt2024/ https://openproceedings.org/html/pages/2...
Publication language:	English
Institution:	School of Business Informatics and Mathematics > Information Systems V: Web-based Systems (Bizer 2012-)
Pre-existing license:	Creative Commons Attribution, Non-Commercial, No Derivatives 4.0 International (CC BY-NC-ND 4.0)
Subject:	004 Computer science, internet
Keywords (English):	entity matching , identity resolution , product matching , benchmarking , evaluation
Abstract:	The difficulty of an entity matching task depends on a combination of multiple factors such as the amount of corner-case pairs, the fraction of entities in the test set that have not been seen during training, and the size of the development set. Current entity matching benchmarks usually represent single points in the space along such dimensions or they provide for the evaluation of matching methods along a single dimension, for instance the amount of training data. This paper presents WDC Products, an entity matching benchmark which provides for the systematic evaluation of matching systems along combinations of three dimensions while relying on real-world data. The three dimensions are (i) amount of corner-cases (ii) generalization to unseen entities, and (iii) development set size (training set plus validation set). Generalization to unseen entities is a dimension not covered by any of the existing English-language benchmarks yet but is crucial for evaluating the robustness of entity matching systems. Inssead of learning how to match entity pairs, entity matching can also be formulated as a multi-class classification task that requires the matcher to recognize individual entities. WDC Products is the first benchmark that provides a pair-wise and a multi-class formulation of the same tasks. We evaluate WDC Products using several state-of-the-art matching systems, including Ditto, HierGAT, and R-SupCon. The evaluation shows that all matching systems struggle with unseen entities to varying degrees. It also shows that for entity matching contrastive learning is more training data efficient compared to cross-encoders.

	Dieser Eintrag ist Teil der Universitätsbibliographie.
	Das Dokument wird vom Publikationsserver der Universitätsbibliothek Mannheim bereitgestellt.
	Dieser Datensatz wurde noch nicht in die Unibibliographie aufgenommen, es ist eine Unveröffentlichte Publikation. Diese Publikation als "Jetzt erschienen" melden.