Measuring progress in dictionary learning for language model interpretability with board game models

URL:	https://openreview.net/forum?id=qzsDKwGJyB
Document Type:	Conference or workshop publication
Year of publication:	2024
Book title:	ICML 2024 Workshop on Mechanistic Interpretability
Page range:	1-17
Conference title:	ICML 2024 Workshop on Mechanistic Interpretability
Location of the conference venue:	Wien, Austria
Date of the conference:	27.07.2024
Publishing house:	OpenReview
Publication language:	English
Institution:	Außerfakultäre Einrichtungen > Institut für Enterprise Systems (InES)
Subject:	004 Computer science, internet
Abstract:	What latent features are encoded in language model (LM) representations? Recent work on training sparse autoencoders (SAEs) to disentangle interpretable features in LM representations has shown significant promise. However, evaluating the quality of these SAEs is difficult because we lack a ground-truth collection of interpretable features that we expect good SAEs to recover. We thus propose to measure progress in interpretable dictionary learning by working in the setting of LMs trained on chess and Othello transcripts. These settings carry natural collections of interpretable features---for example, ``there is a knight on F3''---which we leverage into \textit{supervised} metrics for SAE quality. To guide progress in interpretable dictionary learning, we introduce a new SAE training technique, \textit{p-annealing}, which improves performance on prior unsupervised metrics as well as our new metrics.

Dieser Eintrag ist Teil der Universitätsbibliographie.

BASE: Karvonen, Adam ; Wright, Benjamin ; Rager, Can ; Angell, Rico ; Brinkmann, Jannik ; Smith, Logan Riggs ; Verdun, Claudio Mayrink ; Bau, David ; Marks, Samuel

Google Scholar: Karvonen, Adam ; Wright, Benjamin ; Rager, Can ; Angell, Rico ; Brinkmann, Jannik ; Smith, Logan Riggs ; Verdun, Claudio Mayrink ; Bau, David ; Marks, Samuel

Hits per month over past year

You have found an error? Please let us know about your desired correction here: E-Mail

Show item