Bio

Pepe Ballesteros is a Ph.D. fellow at Digital Visual Studies (University of Zurich). At the age of 22, he received an M.Sc. degree in ’Signal Processing and Machine Learning for Big Data’ from ETSIT-UPM, University of Madrid. His interest in Artificial Intelligence started in the last year of my bachelor’s degree in Telecommunications Systems, where he started my experience as a software engineer at Cirrus Logic International S.L. In collaboration with this company, he researched voice biometrics with AI technologies for his final bachelor thesis, obtaining the maximum qualification. The knowledge acquired in his academic studies motivated him to communicate these methodologies to non-technical audiences, teaching AI-related topics in ’Fundación Univ. Empresa (FUE)’.

 

Winner of the national Spanish contest called ’Impulsa Visión Ayudas a la Investigación’, where the media corporation RTVE finances research projects for master students. The project studied the development of a text generation system to automatically write weather forecasts. In this experience, he had the opportunity to participate in the Mobile World Congress (MWC), where he performed TV and radio interviews to talk about the research project.

 

At the moment, I am interested in ways in which computers can help understand light in art history. I seek mathematical representations of encoded messages in art history through the study of light. I have participated in conferences hosted by the Biblioteca Hertziana and workshops such as BAL-ADRIA (University of Zadar), on the topic of how computational methods might be applied to humanities. Next October, I will present my reflections on practices, methodologies and the future of the field at the IV International Digital Art History Conference in Zagreb.

 

Research Project

Decoding the Expression of Light

The field of Digital Art History (DAH) is characterized by addressing art historical questions with computational methods. At the same time, the interaction between disciplines outlines a key incompatibility between concepts and metrics. Current DAH methodologies remain in the spotlight for art historical criticism, highlighting the struggle of computer models to explain causality and address relevant art historical inquiries. The main challenge of DAH arises from the lack of a common theoretical language between discursive and computational thinking. While the current critique of DAH has seen such incommensurability as the main challenge, this project frames incommensurability as an opportunity.

 

Art history uses language and discursive methodologies to construct models that explain artistic phenomena. This project uses computer language to serve as an alternative construction of such models. Therefore, the aim is not to reproduce approximations of already existing theories and concepts, nor address specific art historical inquiries based on traditional epistemology. If “the limits of my language mean the limits of my world” (L. Wittgenstein), then when the language changes, the world changes: we remain uncertain about what it means to interpret the historical developments of art with computer language. By leveraging computer features, the project attempts to study early modern figurative paintings from large digital collections (e.g. Bibliotheca Hertziana). Throughout the wide spectrum of possible parameters of study, this work is focused on the study of depicted light, where language and discursive methodologies are limited.

 

Depiction of space by means of linear perspective has been widely discussed, whereas pictorial light has received scant attention. Art historians have preferred to emphasize perspective as the major Renaissance achievement rather than light because perspective is more easily defined. Due to its abstract nature, and unlike perspective, accident and intention are not easy to distinguish when describing light with language. If light is considered one of the main instruments for rendering space in paintings, then the description of depicted light features allows the possibility of revealing compositional patterns across paintings. Light is also present as a sign of knowledge, as illumination has been used as a sign of divine communication. The duality of light and form is present as a metaphor for the soul and the body. If light is used to suit a message rather than to fulfill the requirements of the physical world, can we find similar light conditions present in different narratives? Does the meaning of light have a numerical representation? Can we find relationships between light features and the character’s relevance (soul)?

 

Previous work on the analysis of depicted light have been done following a close reading framework, applying statistical methods to understand direction and location of light sources in particular paintings. Other studies have focused on empirical observations to understand human perception of depicted light in paintings. Further research applied relighting methods to approximate the real light conditions present in individual paintings. This project uses Machine Learning feature extraction, and contemporary Deep Learning models for the study of depicted light features following a distant viewing approach. By means of Intrinsic Image Decomposition (IID) and image processing techniques it is possible to describe depicted light conditions (i,e., direction, intensity, diffuseness). To follow a distant view approach, the work utilizes automatic face recognition to analyze light features in depicted faces. Moreover, the extraction of face meshes allows further interpretation of the interaction between face geometry and the use of light. Finally, the development of a computer-based model for the description of depicted light is open to further art historical interpretations related to particular iconographies, schools, and artistic movements.