Prototype of an Unrestricted Image Search of the Crucifixion (2013-2016)

WIN-Kolleg funded by the Heidelberg Academy of Sciences and Humanities; a sub-project of ‘Artistic and Artificial Seeing’

​Peter Bell, Masato Takami, Lisa Dieckmann, Björn Ommer

​Objective of the work: To search within 3620 crucifixion images from the Prometheus Image Archive (Cologne)​

The search algorithm used in the “Passion Search” is designed to eliminate the flaws of the bag-of-words model and should therefore be able to work with arbitrary search requests. With the help of a user-friendly web interface, the user can select particular sections of the image in order to search for them within the dataset. The image can be uploaded by the user or it can be a section of an existing image of the dataset. During the search, positive results can be taken as training examples in order to improve the search results. The algorithm not only enables the detection of identical or very similar image sections but also detects larger variations. The results are then sorted according to the degree of similarity.

Fig 1. The motif itself. A semantically meaningful area of the image is selected through two search boxes on the beams of the cross.

The initial search image selected by the user always appears on the top left (as well as in the bar at the bottom of the page). The results decrease in similarity from left to right and from the first row to the last.
Fig 2. Detection of objects with the same style, composition and medium
Fig 3. Detection of objects from serial artistic production

The image section that was initially searched for appears on the top left of the page and is followed by photographs of the same art work. After these results, the less similar variations of the image section appear as well as related objects. The search results show the more similar results before the less similar ones. This enables the user to understand the search process and determine for him/herself at which point the results become less relevant.

Fig 4. Canonical image sections: The legs of Christ

Although the program has not been set up with an object semantic, some search results still show semantic relationships through the image sections selected by the user.

Fig 5. Canonical motifs such as Christ’s face allow for a comparison that spans a variety of different art epochs.

However, it is interesting to note that most of the results here are from the Middle Ages. (This could be to some extent because of the form of the halo.)
Fig 6. Form and materiality of art objects

The search in Fig 7. is meaningful especially because of the halo and the veil. In comparison to the images of Christ, there is a noticeably larger variety of images. There is also an increase in the variety of image results for the other figures from the Passion.

Fig 7. Depictions of Mary
Fig 8. Selection of an image with geometrical forms

In some cases, semantically meaningful compositions are easily detected and at the same time provide an indication of the epoch, as seen in Fig.9.

Fig 9. Selection of a semantically meaningful composition
Fig 10. Usage of two combined search boxes.