Image with a message : towards detecting non-literal image usages by visual linking


Weiland, Lydia ; Dietz, Laura ; Ponzetto, Simone Paolo



URL: http://www.aclweb.org/anthology/W/W15/W15-2808.pdf
Weitere URL: https://www.cs.cmu.edu/~ark/EMNLP-2015/proceedings...
Dokumenttyp: Konferenzveröffentlichung
Erscheinungsjahr: 2015
Buchtitel: The Workshop on Vision and Language 2015 (VL’15) : Vision and Language Meet Cognitive Systems ; Proceedings ; September 18, 2015 Lisbon, Portugal
Seitenbereich: 40-47
Veranstaltungsdatum: 18.09.2015
Ort der Veröffentlichung: Stroudsburg, Pa.
Verlag: Assoc. for Computational Linguistics
ISBN: 978-1-941643-32-7
Sprache der Veröffentlichung: Englisch
Einrichtung: Fakultät für Wirtschaftsinformatik und Wirtschaftsmathematik > Semantic Web (Juniorprofessur) (Ponzetto 2013-2015)
Fachgebiet: 004 Informatik
Abstract: A key task to understand an image and its corresponding caption is not only to find out what is shown on the picture and described in the text, but also what is the exact relationship between these two elements. The long-term objective of our work is to be able to distinguish different types of relationship, including literal vs. non-literal usages, as well as finegrained non-literal usages (i.e., symbolic vs. iconic). Here, we approach this challenging problem by answering the question: ‘How can we quantify the degrees of similarity between the literal meanings expressed within images and their captions?’. We formulate this problem as a ranking task, where links between entities and potential regions are created and ranked for relevance. Using a Ranking SVM allows us to leverage from the preference ordering of the links, which help us in the similarity calculation for the cases of visual or textual ambiguity, as well as misclassified data. Our experiments show that aggregating different features using a supervised ranker achieves better results than a baseline knowledge-base method. However, much work still lies ahead, and we accordingly conclude the paper with a detailed discussion of a short- and longterm outlook on how to push our work on relationship classification one step further.




Dieser Eintrag ist Teil der Universitätsbibliographie.




Metadaten-Export


Zitation


+ Suche Autoren in

+ Aufruf-Statistik

Aufrufe im letzten Jahr

Detaillierte Angaben



Sie haben einen Fehler gefunden? Teilen Sie uns Ihren Korrekturwunsch bitte hier mit: E-Mail


Actions (login required)

Eintrag anzeigen Eintrag anzeigen