|
Image with a message : towards detecting non-literal image usages by visual linking
Weiland, Lydia
;
Dietz, Laura
;
Ponzetto, Simone Paolo
|
URL:
|
http://www.aclweb.org/anthology/W/W15/W15-2808.pdf
|
|
Additional URL:
|
https://www.cs.cmu.edu/~ark/EMNLP-2015/proceedings...
|
|
Document Type:
|
Conference or workshop publication
|
|
Year of publication:
|
2015
|
|
Book title:
|
The Workshop on Vision and Language 2015 (VL’15) : Vision and Language Meet Cognitive Systems ; Proceedings ; September 18, 2015 Lisbon, Portugal
|
|
Page range:
|
40-47
|
|
Date of the conference:
|
18.09.2015
|
|
Place of publication:
|
Stroudsburg, Pa.
|
|
Publishing house:
|
Assoc. for Computational Linguistics
|
|
ISBN:
|
978-1-941643-32-7
|
|
Publication language:
|
English
|
|
Institution:
|
School of Business Informatics and Mathematics > Semantic Web (Juniorprofessur) (Ponzetto 2013-2015)
|
|
Subject:
|
004 Computer science, internet
|
|
Abstract:
|
A key task to understand an image and its
corresponding caption is not only to find
out what is shown on the picture and described
in the text, but also what is the
exact relationship between these two elements.
The long-term objective of our
work is to be able to distinguish different
types of relationship, including literal
vs. non-literal usages, as well as finegrained
non-literal usages (i.e., symbolic
vs. iconic). Here, we approach this challenging
problem by answering the question:
‘How can we quantify the degrees
of similarity between the literal meanings
expressed within images and their captions?’.
We formulate this problem as a
ranking task, where links between entities
and potential regions are created and
ranked for relevance. Using a Ranking
SVM allows us to leverage from the preference
ordering of the links, which help us
in the similarity calculation for the cases
of visual or textual ambiguity, as well as
misclassified data. Our experiments show
that aggregating different features using a
supervised ranker achieves better results
than a baseline knowledge-base method.
However, much work still lies ahead, and
we accordingly conclude the paper with a
detailed discussion of a short- and longterm
outlook on how to push our work on
relationship classification one step further.
|
 | Dieser Eintrag ist Teil der Universitätsbibliographie. |
 Search Authors in
BASE:
Weiland, Lydia
;
Dietz, Laura
;
Ponzetto, Simone Paolo
Google Scholar:
Weiland, Lydia
;
Dietz, Laura
;
Ponzetto, Simone Paolo
ORCID:
Weiland, Lydia, Dietz, Laura and Ponzetto, Simone Paolo  ORCID: https://orcid.org/0000-0001-7484-2049
You have found an error? Please let us know about your desired correction here: E-Mail
Actions (login required)
 |
Show item |
|
|
