Cuneiform Sign Detection

Cuneiform clay tablets are the oldest written records of humankind providing invaluable insights into our ancient history. Our group has developed a cuneiform sign detector that can localize and classify cuneiform signs in images of clay tablets by training a deep learning-based model without requiring manual sign annotations.

Our approach leverages existing transliterations (sign-by-sign representation of tablet content in Latin script) in order to supervise the sign detector training. Since a transliteration does not provide the absolute position of cuneiform signs in a tablet image, our method learns how to align hundreds of tablet images and corresponding transliterations in parallel in order to generate localized annotations for detector training.

This is significant because the proposed method shows how to perform weakly supervised learning in a difficult fine-grained detection setting. To the best of our knowledge, this is the first work to demonstrate strong sign detection for over a hundred sign code classes evaluated on a large and diverse collection of tablet images. Since our method is applicable to cuneiform script of different periods, it enables large-scale digital analysis of cuneiform script in the field of Assyriology, thus helping to more efficiently open up the treasures of human civilization. To directly support Assyriologists in their analysis of cuneiform texts, we also present a web application of the cuneiform sign detector, thus making it easily applicable.

Resources

Web Application

Code & Trained Models

Dataset

Materials and Methods

Full details about the method, its implementation, and a detailed performance analysis can be found on the website of our article at PLOS ONE.

We provide the dataset and the implementation of our method in two public repositories. The cuneiform sign detection dataset is available in this data repo and the code and pre-trained models of the cuneiform sign detector are available in this code repo. In a third repository, we also provide the code for the web application of the cuneiform sign detector which is available in this web app repo. 

For demonstration purposes, we also host an instance of the web application as a demo system. If you would like to test the web application, please send an email to this address to request login data. 

Publication

@article{7029,

title = {Deep learning of cuneiform sign detection with weak supervision using transliteration alignment},

author = {Tobias Dencker and Pablo Klinkisch and Stefan M Maul and Björn Ommer},

url = {https://hci.iwr.uni-heidelberg.de/compvis/projects/cuneiform

https://ommer-lab.com/wp-content/uploads/2021/10/Deep-Learning-of-Cuneiform-Sign-Detection-with-Weak-Supervivion-Using-Transliteration-Alignment.pdf

},

doi = {https://doi.org/10.1371/journal.pone.0243039},

year  = {2020},

date = {2020-01-01},

urldate = {2020-01-01},

journal = {PLoS ONE},

volume = {15},

chapter = {1-21},

abstract = {The cuneiform script provides a glimpse into our ancient history. However, reading age-old clay tablets is time-consuming and requires years of training. To simplify this process, we propose a deep-learning based sign detector that locates and classifies cuneiform signs in images of clay tablets. Deep learning requires large amounts of training data in the form of bounding boxes around cuneiform signs, which are not readily available and costly to obtain in the case of cuneiform script. To tackle this problem, we make use of existing transliterations, a sign-by-sign representation of the tablet content in Latin script. Since these do not provide sign localization, we propose a weakly supervised approach: We align tablet images with their corresponding transliterations to localize the transliterated signs in the tablet image, before using these localized signs in place of annotations to re-train the sign detector. A better sign detector in turn boosts the quality of the alignments. We combine these steps in an iterative process that enables training a cuneiform sign detector from transliterations only. While our method works weakly supervised, a small number of annotations further boost the performance of the cuneiform sign detector which we evaluate on a large collection of clay tablets from the Neo-Assyrian period. To enable experts to directly apply the sign detector in their study of cuneiform texts, we additionally provide a web application for the analysis of clay tablets with a trained cuneiform sign detector.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}