Visual Search Tool

The Computer Vision group develops automatic methods to process and evaluate large image collections, including a retrieval technique for objects, which was implemented in a publicly accessible visual search tool. The system was designed in collaboration with computer scientists and art historians to assist with art historical research and considered requirements imposed by humanities’ research. The following aspects were relevant: user interaction, easy/comprehensive navigation, allow to evaluate image collections of different media and genre etc and search for multiple objects, thus enabling an iconographic analysis. Eventually, the group has applied this search engine to various image collections and demonstrated its efficiency.

Interface for Visual Search

The system holds various datasets, which are uploaded by users. A visual search is performed within these collections based on single and multiple object queries (up to five), as marked by the user. A search process is triggered, where the underlying algorithm retrieves identical as well as similar regions to the query. Results are displayed in a new window with decreasing similarity. In order to refine the classifier and improve the precision of results, the group implemented a feedback system, where the user selects positive and negative retrievals. In contrast to other search tools, the presented system solely operates on visual input, which is especially useful when we work with unlabelled data.

The visual search tool and its function have been developed with regards to demands, imposed by humanities’ research (tasks, usability etc). It has been successfully applied to diverse image corpora including portraits, medieval manuscripts, architectural drawings, paintings and photographs. These projects, some of which are described in the publications listed below, demonstrate the efficiency of said tool.

Get more information

More information on an unrestricted image search based on user-selected exemplars can be found here.

Selected Publications


Ufer, Nikolai; Simon, Max; Lang, Sabine; Ommer, Björn

Large-scale interactive retrieval in art collections using multi-style feature aggregation Journal Article

In: PLoS ONE, vol. 16, no. 11, 2021.

Links | BibTeX


Lang, Sabine; Ommer, Björn

Attesting Similarity: Supporting the Organization and Study of Art Image Collections with Computer Vision Journal Article

In: Digital Scholarship in the Humanities, Oxford University Press, vol. 33, no. 4, pp. 845-856, 2018.

Links | BibTeX

Lang, Sabine; Ommer, Björn

Reconstructing Histories: Analyzing Exhibition Photographs with Computational Methods Journal Article

In: Arts, Computational Aesthetics, vol. 7, 64, 2018.

Links | BibTeX


Takami, M.; Bell, P.; Ommer, Björn

An Approach to Large Scale Interactive Retrieval of Cultural Heritage Conference

Eurographics Workshop on Graphics and Cultural Heritage, The Eurographics Association, 2014.

Links | BibTeX