Explainable AI

Deep learning has significantly improved the expressiveness of representations. However, present research still fails to understand why and how they work and cannot reliably predict when they fail. Moreover, the different characteristics of our physical world are commonly intermingled, making it impossible to study them individually. We incorporate novel paradigms for disentangling multiple object characteristics and present interpretable models to translate arbitrary network representations into semantically meaningful, interpretable concepts. We also obtain disentangled generative models that explain their latent representations by synthesis while being able to alter different object characteristics individually.

2021

Islam, Md Amirul; Kowal, Matthew; Esser, Patrick; Jia, Sen; Ommer, Björn; Derpanis, Konstantinos G; Bruce, Neil

Shape or Texture: Understanding Discriminative Features in CNNs Conference

International Conference on Learning Representations (ICLR), 2021.

Links | BibTeX

2020

A Disentangling Invertible Interpretation Network for Explaining Latent Representations

Esser, Patrick; Rombach, Robin; Ommer, Björn

A Disentangling Invertible Interpretation Network for Explaining Latent Representations Conference

Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.

Abstract | Links | BibTeX

Esser, Patrick; Rombach, Robin; Ommer, Björn

A Note on Data Biases in Generative Models Conference

NeurIPS 2020 Workshop on Machine Learning for Creativity and Design, 2020.

Abstract | Links | BibTeX

Making Sense of CNNs: Interpreting Deep Representations & Their Invariances with INNs

Rombach, Robin; Esser, Patrick; Ommer, Björn

Making Sense of CNNs: Interpreting Deep Representations & Their Invariances with INNs Conference

IEEE European Conference on Computer Vision (ECCV), 2020.

Abstract | Links | BibTeX

Network Fusion for Content Creation with Conditional INNs

Rombach, Robin; Esser, Patrick; Ommer, Björn

Network Fusion for Content Creation with Conditional INNs Conference

CVPRW 2020 (AI for Content Creation), 2020.

Abstract | Links | BibTeX

Network-to-Network Translation with Conditional Invertible Neural Networks

Rombach, Robin; Esser, Patrick; Ommer, Björn

Network-to-Network Translation with Conditional Invertible Neural Networks Conference

Neural Information Processing Systems (NeurIPS) (Oral), 2020.

Abstract | Links | BibTeX

2019

Content and Style Disentanglement for Artistic Style Transfer

Kotovenko, Dmytro; Sanakoyeu, Artsiom; Lang, Sabine; Ommer, Björn

Content and Style Disentanglement for Artistic Style Transfer Conference

Proceedings of the Intl. Conf. on Computer Vision (ICCV), 2019.

Links | BibTeX

Unsupervised Part-Based Disentangling of Object Shape and Appearance

Lorenz, Dominik; Bereska, Leonard; Milbich, Timo; Ommer, Björn

Unsupervised Part-Based Disentangling of Object Shape and Appearance Conference

Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (Oral + Best paper finalist: top 45 / 5160 submissions), 2019.

Links | BibTeX

Unsupervised Robust Disentangling of Latent Characteristics for Image Synthesis

Esser, Patrick; Haux, Johannes; Ommer, Björn

Unsupervised Robust Disentangling of Latent Characteristics for Image Synthesis Conference

Proceedings of the Intl. Conf. on Computer Vision (ICCV), 2019.

Abstract | Links | BibTeX

2018

A Variational U-Net for Conditional Appearance and Shape Generation

Esser, Patrick; Sutter, Ekaterina; Ommer, Björn

A Variational U-Net for Conditional Appearance and Shape Generation Conference

Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (short Oral), 2018.

Abstract | Links | BibTeX