Add-remove-confirm: crowdsourcing synset cleansing

DOI:	https://doi.org/10.1109/ICAICT.2015.7338534
URL:	https://www.researchgate.net/publication/308830410...
Additional URL:	http://ieeexplore.ieee.org/document/7338534/
Document Type:	Conference or workshop publication
Year of publication:	2015
Book title:	2015 9th International Conference on Application of Information and Communication Technologies (AICT 2015) : Rostov-on-Don, Russia, 14 - 16 October 2015
Page range:	143-147
Conference title:	2015 9th International Conference on Application of Information and Communication Technologies
Location of the conference venue:	Rostov-on-Don, Russia
Date of the conference:	October 14-16, 2015
Publisher:	Adamov, Abzetdin
Place of publication:	Piscataway, NJ ; Red Hook, NJ
Publishing house:	IEEE ; Curran
ISBN:	978-1-4673-6857-5 , 978-1-4673-6855-1
Publication language:	English
Institution:	School of Business Informatics and Mathematics > Wirtschaftsinformatik III (Ponzetto 2016-)
Subject:	004 Computer science, internet
Abstract:	Thesaurus is a crucial resource for many natural language processing and artificial intelligence problems, which require common sense reasoning. It is becoming highly topical to put special effort to ensure the high quality of synsets when a thesaurus is created collaboratively by non-expert annotators. This paper proposes Add-Remove-Confirm, a novel workflow for crowdsourcing synset cleansing. The present workflow has been empirically evaluated using a Russian thesaurus created through crowdsourcing showing that it does improve the synset quality as according to the expert assessment with high level of agreement.