Deep Metric and Representation Learning

To understand visual content, computers need to learn what makes images similar. This similarity learning directly implies a representation of the visual content that captures the inherent structure of the data. We present several approaches that can be applied on top of arbitrary deep metric learning methods and various network architectures. Key issues that these works tackle include improving generalization and transfer to novel data, shared feature learning, and adaptive sampling strategies based on reinforcement learning to effectively utilize large amounts of training data.

Following, we provide a (selective) overview of our research on visual similarity learning. For a comprehensive list, please visit our publication page.

Talk given in July 2021

Selected Publications

2022

Milbich, Timo; Roth, Karsten; Brattoli, Biagio; Ommer, Björn

Sharing Matters for Generalization in Deep Metric Learning Journal Article

In: IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2022.

Links | BibTeX

2021

Milbich, Timo; Roth, Karsten; Sinha, Samarth; Schmidt, Ludwig; Ghassemi, Marzyeh; Ommer, Björn

Characterizing Generalization under Out-Of-Distribution Shifts in Deep Metric Learning Conference

Neural Information Processing Systems (NeurIPS), 2021.

Links | BibTeX

Sanakoyeu, Artsiom; Ma, Pingchuan; Tschernezki, Vadim; Ommer, Björn

Improving Deep Metric Learning by Divide and Conquer Journal Article

In: IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.

Links | BibTeX

Roth, Karsten; Milbich, Timo; Ommer, Björn; Cohen, Joseph Paul; Ghassemi, Marzyeh

S2SD: Simultaneous Similarity-based Self-Distillation for Deep Metric Learning Conference

Proceedings of International Conference on Machine Learning (ICML), 2021.

Links | BibTeX

2020

Milbich, Timo; Roth, Karsten; Bharadhwaj, Homanga; Sinha, Samarth; Bengio, Yoshua; Ommer, Björn; Cohen, Joseph Paul

DiVA: Diverse Visual Feature Aggregation for Deep Metric Learning Conference

IEEE European Conference on Computer Vision (ECCV), 2020.

Links | BibTeX

Roth, Karsten; Milbich, Timo; Sinha, Samarth; Gupta, Prateek; Ommer, Björn; Cohen, Joseph Paul

Revisiting Training Strategies and Generalization Performance in Deep Metric Learning Conference

International Conference on Machine Learning (ICML), 2020.

Links | BibTeX

Milbich, Timo; Roth, Karsten; Ommer, Björn

PADS: Policy-Adapted Sampling for Visual Similarity Learning Conference

Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, 2020.

Links | BibTeX

Milbich, Timo; Ghori, Omair; Ommer, Björn

Unsupervised Representation Learning by Discovering Reliable Image Relations Journal Article

In: Pattern Recognition, vol. 102, 2020.

Links | BibTeX

2019

Sanakoyeu, A.; Tschernezki, V.; Büchler, Uta; Ommer, Björn

Divide and Conquer the Embedding Space for Metric Learning Conference

Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.

Links | BibTeX

Brattoli, Biagio; Roth, Karsten; Ommer, Björn

MIC: Mining Interclass Characteristics for Improved Metric Learning Conference

Proceedings of the Intl. Conf. on Computer Vision (ICCV), 2019.

Links | BibTeX

2018

Sanakoyeu, A.; Bautista, Miguel; Ommer, Björn

Deep Unsupervised Learning of Visual Similarities Journal Article

In: Pattern Recognition, vol. 78, 2018.

Abstract | Links | BibTeX

Büchler, Uta; Brattoli, Biagio; Ommer, Björn

Improving Spatiotemporal Self-Supervision by Deep Reinforcement Learning Conference

Proceedings of the European Conference on Computer Vision (ECCV), (UB and BB contributed equally), Munich, Germany, 2018.

Abstract | Links | BibTeX

2017

Bautista, Miguel; Sanakoyeu, A.; Ommer, Björn

Deep Unsupervised Similarity Learning using Partially Ordered Sets Conference

The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.

Links | BibTeX

Milbich, Timo; Bautista, Miguel; Sutter, Ekaterina; Ommer, Björn

Unsupervised Video Understanding by Reconciliation of Posture Similarities Conference

Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017.

Links | BibTeX

2016

Bautista, Miguel; Sanakoyeu, A.; Sutter, E.; Ommer, Björn

CliqueCNN: Deep Unsupervised Exemplar Learning Conference

Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS), MIT Press, Barcelona, 2016.

Abstract | Links | BibTeX