|
Measuring progress in dictionary learning for language model interpretability with board game models
Karvonen, Adam
;
Wright, Benjamin
;
Rager, Can
;
Angell, Rico
;
Brinkmann, Jannik
;
Smith, Logan Riggs
;
Verdun, Claudio Mayrink
;
Bau, David
;
Marks, Samuel
|
URL:
|
https://openreview.net/forum?id=qzsDKwGJyB
|
|
Dokumenttyp:
|
Konferenzveröffentlichung
|
|
Erscheinungsjahr:
|
2024
|
|
Buchtitel:
|
ICML 2024 Workshop on Mechanistic Interpretability
|
|
Seitenbereich:
|
1-17
|
|
Veranstaltungstitel:
|
ICML 2024 Workshop on Mechanistic Interpretability
|
|
Veranstaltungsort:
|
Wien, Austria
|
|
Veranstaltungsdatum:
|
27.07.2024
|
|
Verlag:
|
OpenReview
|
|
Sprache der Veröffentlichung:
|
Englisch
|
|
Einrichtung:
|
Außerfakultäre Einrichtungen > Institut für Enterprise Systems (InES)
|
|
Fachgebiet:
|
004 Informatik
|
|
Abstract:
|
What latent features are encoded in language model (LM) representations? Recent work on training sparse autoencoders (SAEs) to disentangle interpretable features in LM representations has shown significant promise. However, evaluating the quality of these SAEs is difficult because we lack a ground-truth collection of interpretable features that we expect good SAEs to recover. We thus propose to measure progress in interpretable dictionary learning by working in the setting of LMs trained on chess and Othello transcripts. These settings carry natural collections of interpretable features---for example, ``there is a knight on F3''---which we leverage into \textit{supervised} metrics for SAE quality. To guide progress in interpretable dictionary learning, we introduce a new SAE training technique, \textit{p-annealing}, which improves performance on prior unsupervised metrics as well as our new metrics.
|
 | Dieser Eintrag ist Teil der Universitätsbibliographie. |
 Suche Autoren in
BASE:
Karvonen, Adam
;
Wright, Benjamin
;
Rager, Can
;
Angell, Rico
;
Brinkmann, Jannik
;
Smith, Logan Riggs
;
Verdun, Claudio Mayrink
;
Bau, David
;
Marks, Samuel
Google Scholar:
Karvonen, Adam
;
Wright, Benjamin
;
Rager, Can
;
Angell, Rico
;
Brinkmann, Jannik
;
Smith, Logan Riggs
;
Verdun, Claudio Mayrink
;
Bau, David
;
Marks, Samuel
Sie haben einen Fehler gefunden? Teilen Sie uns Ihren Korrekturwunsch bitte hier mit: E-Mail
Actions (login required)
 |
Eintrag anzeigen |
|
|
