Abstract

Title: Deep Learning of Style – From Stylistic Concepts to Computational Metrics, and Back?​​​​​​

By Eva Cetinic

The notion of style, in all its fuzziness, represents the building (and also stumbling) block of interdisciplinarity in computational analysis of art. Within the scope of various challenges that exist in the intersection of computer vision, deep learning and art history, the concept of style appears to be the one that most commonly highlights the discrepancies between disciplines. In the broader computer vision community, “style” is often understood simply as the texturaldivergence from photorealistic depictions, something that can be disentangled from the “content” (the objects or the scene that are depicted). In computer vision research which is more specialized in working with data coming from the domain of art, style is most commonly addressed in the context of automated classification. Here “style” becomes the label that is associated with a particular item in large datasets, e.g. “impressionism”, “expressionism”, “realism”, etc. Art historians know that an artwork can often simultaneously belong to more than one category and that these kinds of categorizations are in themselves often a very ambiguous “ground-truth”. However, in order to start from somewhere, computationalapproaches have to use conventionalized typifications and rough approximations of complex concepts such as style. Art represents a prolific source of data for challenging computational image analysis at the highest level of complexity. However, improvement of performance metrics for a particular classification task or model, which is often considered the end goal from the computational point of view, mostly bear little value for domain specific knowledge production in art history. In order to converge from simplistic appropriations towards more mature interdisciplinary knowledge production, in recent years an increased number of research works focused on exploring how to quantify theoretical concepts relevant for art history, particularly those related to formal analysis of style. However, within contemporary art history research, interest in topics related to formal analysis of style is scarce and it remains questionable if computational approaches can provide novel perspectives and help revive this interest. In order to overcome the manifold interdisciplinary challenges in understanding and analyzing style, it is of great importance to communicate the various aspects of different approaches. This is particularly relevant in the context of deep learning, where deep neural network models comprise huge parametric spaces trained on large datasets and therefore suffer from the known “black box” problem. For example, if a model is trained to classify “style”, what does it actually learn, what are the distinct features and can this model be used for other purposes? How can those models be employed to analyze data, but also how can data be used to explain the models? What kind of datasets are used to train such models and what limitations do they impose, i.e. what kind of biases are integrated and propagated through the use of such models? And what are the long term implications of the data hegemony of a few elite institutions which can afford to annotate large datasets and develop complex style-trained models which become integrated in retrieval systems or commercial image-editing software? The proposed presentation aims to address those questions while providing an overview of different research examples of deep learning-based computational analysis of style.

Seminar

Styles  Revisited: From Iconology to Digital Image Studies
2022-2023 Artl@s/Visual Contagions Research Seminar.

Organizers: Béatrice Joyeux-Prunel (UNIGE), Catherine Dossin (Purdue University), and Nicola Carboni (UNIGE)