Fine-grained evaluation of rule- and embedding-based systems for knowledge graph completion

Meilicke, Christian ; Fink, Manuel ; Wang, Yanjie ; Ruffinelli, Daniel ; Gemulla, Rainer ; Stuckenschmidt, Heiner

DOI:	https://doi.org/10.1007/978-3-030-00671-6_1
URL:	https://link.springer.com/chapter/10.1007/978-3-03...
Additional URL:	https://link.springer.com/book/10.1007/978-3-030-0...
Document Type:	Conference or workshop publication
Year of publication:	2018
Book title:	The Semantic Web – ISWC 2018 : 17th International Semantic Web Conference, Monterey, CA, USA, October 8–12, 2018, Proceedings, Part I
The title of a journal, publication series:	Lecture Notes in Computer Science
Page range:	3-20
Conference title:	ISWC 2018
Location of the conference venue:	Monterey, CA
Date of the conference:	October 8-12, 2018
Publisher:	Vrandečić, Denny
Place of publication:	Berlin [u.a.]
Publishing house:	Springer
ISBN:	978-3-030-00670-9 , 978-3-030-00672-3 , 978-3-030-00671-6
ISSN:	0302-9743 , 1611-3349
Publication language:	English
Institution:	School of Business Informatics and Mathematics > Practical Computer Science II: Artificial Intelligence (Stuckenschmidt 2009-)
Subject:	004 Computer science, internet
Keywords (English):	rule learning , embeddings , knowledge graph completion , link prediction
Abstract:	Over the recent years embeddings have attracted increasing research focus as a means for knowledge graph completion. Similarly, rule-based systems have been studied for this task in the past as well. What is missing from existing works so far, is a common evaluation that includes more than one type of method. We close this gap by comparing representatives of both types of systems in a frequently used evaluation format. Leveraging the explanatory qualities of rule-based systems, we present a fine-grained evaluation scenario that gives insight into characteristics of the most popular datasets and points out the different strengths and shortcomings of the examined approaches. Our results show that models such as TransE, RESCAL or HolE have problems in solving certain types of completion tasks that can be solved by a rule-based approach with high precision. At the same time there are other completion tasks that are difficult for rule-based systems. Motivated by these insights we combine both families of approaches via ensemble learning. The results support our assumption that the two methods can complement each other in a beneficial way.