A deep learning approach to fight illicit trafficking of antiquities using artefact instance classification

Winterbottom, Thomas; Leone, Anna; Al Moubayed, Noura

Abstract

We approach the task of detecting the illicit movement of cultural heritage from a machine learning perspective by presenting a framework for detecting a known artefact in a new and unseen image. To this end, we explore the machine learning problem of instance classification for large archaeological images datasets, i.e. where each individual object (instance) is itself a class that all of the multiple images of that object belongs. We focus on a wide variety of objects in the Durham Oriental Museum with which we build a dataset with over 24,502 images of 4332 unique object instances. We experiment with state-of-the-art convolutional neural network models, the smaller variations of which are suitable for deployment on mobile applications. We find the exact object instance of a given image can be predicted from among 4332 others with ~ 72% accuracy, showing how effectively machine learning can detect a known object from a new image. We demonstrate that accuracy significantly improves as the number of images-per-object instance increases (up to ~ 83%), with an ensemble of classifiers scoring as high as 84%. We find that the correct instance is found in the top 3, 5, or 10 predictions of our best models ~ 91%, ~ 93%, or ~ 95% of the time respectively. Our findings contribute to the emerging overlap of machine learning and cultural heritage, and highlights the potential available to future applications and research.

Citation

Winterbottom, T., Leone, A., & Al Moubayed, N. (2022). A deep learning approach to fight illicit trafficking of antiquities using artefact instance classification. Scientific Reports, 12(1), Article 13468. https://doi.org/10.1038/s41598-022-15965-2

• Journal Article Type: Article
• Acceptance Date: Jul 1, 2022
• Online Publication Date: Aug 5, 2022
• Publication Date: 2022
• Deposit Date: Aug 18, 2022
• Publicly Available Date: Aug 18, 2022
• Journal: Scientific Reports
• Electronic ISSN: 2045-2322
• Publisher: Nature Research
• Peer Reviewed: Peer Reviewed
• Volume: 12
• Issue: 1
• Article Number: 13468
• DOI: https://doi.org/10.1038/s41598-022-15965-2
• Public URL: https://durham-repository.worktribe.com/output/1194110