PhD defense of M. Abdelhak BELHI

Date: 
Wednesday 15 July 2020

M. Abdelhak BELHI will defend his PhD Wednesday July 15th at 9AM.

Thesis Title: Digital Cultural Heritage Preservation: Enrichment and Reconstruction based on Hierarchical Multimodal CNNs and Image Inpainting Approaches

Jury:

  • Nada MATTA, Professeur des Universités, Université de Technologie de Troyes, Rapporteur
  • Dominique MICHELUCCI, Professeur des Universités, Université de Bourgogne, Rapporteur
  • Salah SADOU, Professeur des Universités, Université de Bretagne-Sud, Examinateur
  • Tewfik ZIADI, Maître de Conférences-HDR, Sorbonne Université, Examinateur
  • Abdelaziz BOURAS, Professeur des Universités, Université Lumière Lyon 2, Directeur de thèse
  • Sebti FOUFOU, Professeur des Universités, Université de Bourgogne, Co-encadrant de thèse

Abstract: Cultural heritage plays an important role in defining the identity of a society. Long-term physical preservation of cultural heritage remains risky and can lead to multiple problems related to destruction and accidental damage. Digital technologies such as photography and 3D scanning provided new alternatives for digital preservation. However, adapting them to the context of cultural heritage is a challenging task. In fact, fully digitizing cultural assets (visually and historically) is only easy when it comes to assets that are in a good physical shape and all their data is at possession (fully annotated). However, in the real-world, many assets suffer from physical degradation and information loss. Usually, to annotate and curate these assets, heritage institutions need the help of art specialists and historians. This process is tedious, involves considerable time and financial resources, and can often be inaccurate.

Our work focuses on the cost-effective preservation of cultural heritage through advanced machine learning methods. The aim is to provide a technical framework for the enrichment phase of the cultural heritage digital preservation/curation process. Through this thesis, we propose new methods to improve the process of cultural heritage preservation. Our challenges are mainly related to the annotation and enrichment of cultural objects suffering from missing and incomplete data (annotations and visual data) which is often considered ineffective when performed manually. Thus, we propose approaches based on machine learning and deep learning to tackle these challenges. These approaches consist of the automatic completion of missing cultural data. We mainly focus on two types of missing data: textual data (metadata) and visual data.

The first stage is mainly related to the annotation and labeling of cultural objects using deep learning. We have proposed approaches, that take advantage of cultural objects’ visual features as well as partially available textual annotations, to perform an effective classification. (i) the first approach is related to the Hierarchical Classification of Objects to better meet the metadata requirements of each cultural object type and increase the classification algorithm performance. (ii) the second proposed approach is dedicated to the Multimodal Classification of cultural objects where any object can be represented, during classification, with a subset of available metadata in addition to its visual capture. The second stage considers the lack of visual information when dealing with incomplete and damaged cultural objects. In this case, we proposed an approach based on deep learning through generative models and image data clustering to optimize the image completion process of damaged cultural heritage objects. For our experiments, we collected a large database of cultural objects. We chose to use fine-art paintings in our tests and validations as they were the best in terms of annotations quality.

Keywords: Preservation of Cultural Objects, Deep Learning, Multimodal Classification, Image Completion.