Learning disentangled discrete representations

Friede, David ; Reimers, Christian ; Stuckenschmidt, Heiner ; Niepert, Mathias

DOI: https://doi.org/10.1007/978-3-031-43421-1_35
URL: https://link.springer.com/chapter/10.1007/978-3-03...
Document Type: Conference or workshop publication
Year of publication: 2023
Book title: Machine learning and knowledge discovery in databases : research track : European conference, EMCL PKDD 2023, Turin, Italy, september 18-22,2023, proceedings, part IV
The title of a journal, publication series: Lecture Notes in Computer Science
Volume: 14172
Page range: 593-609
Conference title: ECML PKDD 2023, European Conference on Machine Learning and Data Mining
Location of the conference venue: Torino, Italy
Date of the conference: 18.-22.09.2023
Publisher: Koutra, Danai ; Plant, Claudia ; Gomez Rodriguez, Manuel ; Baralis, Elena ; Bonchi, Francesco
Place of publication: Berlin [u.a.]
Publishing house: Springer
ISBN: 978-3-031-43420-4 , 978-3-031-43421-1
ISSN: 0302-9743 , 1611-3349
Related URLs:
Publication language: English
Institution: School of Business Informatics and Mathematics > Practical Computer Science II: Artificial Intelligence (Stuckenschmidt 2009-)
Subject: 004 Computer science, internet
Keywords (English): categorical VAE , disentanglement
Abstract: Recent successes in image generation, model-based reinforcement learning, and text-to-image generation have demonstrated the empirical advantages of discrete latent representations, although the reasons behind their benefits remain unclear. We explore the relationship between discrete latent spaces and disentangled representations by replacing the standard Gaussian variational autoencoder (VAE) with a tailored categorical variational autoencoder. We show that the underlying grid structure of categorical distributions mitigates the problem of rotational invariance associated with multivariate Gaussian distributions, acting as an efficient inductive prior for disentangled representations. We provide both analytical and empirical findings that demonstrate the advantages of discrete VAEs for learning disentangled representations. Furthermore, we introduce the first unsupervised model selection strategy that favors disentangled representations.

Dieser Eintrag ist Teil der Universitätsbibliographie.

Metadata export


+ Search Authors in

+ Page Views

Hits per month over past year

Detailed information

You have found an error? Please let us know about your desired correction here: E-Mail

Actions (login required)

Show item Show item