Visual Search Tool - Computer Vision & Learning Group

The Computer Vision group develops automatic methods to process and evaluate large image collections, including a retrieval technique for objects, which was implemented in a publicly accessible visual search tool. The system was designed in collaboration with computer scientists and art historians to assist with art historical research and considered requirements imposed by humanities’ research. The following aspects were relevant: user interaction, easy/comprehensive navigation, allow to evaluate image collections of different media and genre etc and search for multiple objects, thus enabling an iconographic analysis. Eventually, the group has applied this search engine to various image collections and demonstrated its efficiency.

Interface for Visual Search

The system holds various datasets, which are uploaded by users. A visual search is performed within these collections based on single and multiple object queries (up to five), as marked by the user. A search process is triggered, where the underlying algorithm retrieves identical as well as similar regions to the query. Results are displayed in a new window with decreasing similarity. In order to refine the classifier and improve the precision of results, the group implemented a feedback system, where the user selects positive and negative retrievals. In contrast to other search tools, the presented system solely operates on visual input, which is especially useful when we work with unlabelled data.

The visual search tool and its function have been developed with regards to demands, imposed by humanities’ research (tasks, usability etc). It has been successfully applied to diverse image corpora including portraits, medieval manuscripts, architectural drawings, paintings and photographs. These projects, some of which are described in the publications listed below, demonstrate the efficiency of said tool.