|
Evaluating knowledge generation and self-refinement strategies for LLM-based column type annotation
Korini, Keti
;
Bizer, Christian
|
DOI:
|
https://doi.org/10.1007/978-3-032-05281-0_8
|
|
URL:
|
https://link.springer.com/chapter/10.1007/978-3-03...
|
|
Weitere URL:
|
https://www.researchgate.net/publication/395684599...
|
|
Dokumenttyp:
|
Konferenzveröffentlichung
|
|
Erscheinungsjahr:
|
2026
|
|
Buchtitel:
|
Advances in Databases and Information Systems : 29th European Conference, ADBIS 2025 Tampere, Finland, September 23–26, 2025, proceedings
|
|
Titel einer Zeitschrift oder einer Reihe:
|
Lecture Notes in Computer Science
|
|
Band/Volume:
|
16043
|
|
Seitenbereich:
|
111-127
|
|
Veranstaltungstitel:
|
ADBIS 2025
|
|
Veranstaltungsort:
|
Tampere, Finland
|
|
Veranstaltungsdatum:
|
23.-26.09.2025
|
|
Herausgeber:
|
Chrysanthis, Panos K.
;
Nørvåg, Kjetil
;
Stefanidis, Kostas
;
Zhang, Zheying
|
|
Ort der Veröffentlichung:
|
Berlin [u.a.]
|
|
Verlag:
|
Springer
|
|
ISBN:
|
978-3-032-05280-3 , 978-3-032-05281-0
|
|
ISSN:
|
0302-9743 , 1611-3349
|
|
Verwandte URLs:
|
|
|
Sprache der Veröffentlichung:
|
Englisch
|
|
Einrichtung:
|
Fakultät für Wirtschaftsinformatik und Wirtschaftsmathematik > Information Systems V: Web-based Systems (Bizer 2012-)
|
|
Fachgebiet:
|
004 Informatik
|
|
Freie Schlagwörter (Englisch):
|
table annotation , Large Language Models , column type annotation , knowledge generation prompting , fine-tuning , error-based self-refinement
|
|
Abstract:
|
Understanding the semantics of columns in relational tables is an important pre-processing step for indexing data lakes in order to provide rich data search. An approach to establishing such understanding is column type annotation (CTA) where the goal is to annotate table columns with terms from a given vocabulary. This paper experimentally compares knowledge generation and self-refinement strategies for LLM-based CTA. The strategies include using LLMs to generate term definitions, error-based refinement of term definitions, and fine-tuning using examples and term definitions. We evaluate these strategies along two dimensions: effectiveness measured as F1 performance and efficiency measured in terms of token usage and cost. Our experiments show that using training data to generate label definitions outperforms using the same data as demonstrations for in-context learning for two out of three datasets using OpenAI models. The experiments further show that using the LLMs to refine label definitions brings an average increase of 3.9% F1 in most setups compared to the performance of the non-refined definitions. Combining fine-tuned models with self-refined term definitions results in the overall highest performance for gpt-4o, outperforming zero-shot prompting fine-tuned models by at least 3% in F1. The cost analysis shows that self-refinement via prompting is more cost-efficient than fine-tuning for use cases requiring smaller amounts of tables to be annotated.
|
 | Dieser Eintrag ist Teil der Universitätsbibliographie. |
 Suche Autoren in
Sie haben einen Fehler gefunden? Teilen Sie uns Ihren Korrekturwunsch bitte hier mit: E-Mail
Actions (login required)
 |
Eintrag anzeigen |
|
|
