Rotate your phone to navigate through the site properly
Four axes form the basic structure for the research program:
The first research focus is set on computer vision’s potential contribution to the history of art and architecture. As the photographic collection of the BHMPI is currently being digitized at a fast rate, and a massive number of art-related research data is being pooled by several partnering institutions (see 3.3), new methodologies will be developed to read, order, and represent art historical visual material. Supervised and semi-supervised machine learning will be possible thanks to the high-quality metadata of existing visual records, automatically predicting metadata for the new unlabeled images – whilst unsupervised methods will also be useful to organize and search the image-datasets through elements not present in the metadata (style, iconography, gesture). These will combine a number of existing techniques, applied to different material (face recognition, style recognition, gesture recognition, sketch-based image retrieval) with new problems in computer vision (architectural pattern recognition, artistic attribute detection). Further annotations will be collected during users’ and scholars’ employment of these prototype systems, improving the quality of both the underlying data and – through online machine learning – future models: such a mechanism necessitates the rapid technology transfer of subsequent computer vision prototypes to the BHMPI’s online photo archive interface. As well as defining new problems in computer vision, the annotations created in such a process can be offered to the computer vision and pattern recognition community, providing new machine learning databases and baselines for relevant computer science research communities, such as ECCV-VISART and ACM Expressive.
The second focal point uses and develops natural language processing techniques within the methodology of Distant Reading. As the digitization initiative of the BHMPI library will then have added special collections (see 3.1.1), the doctoral students will be able to develop new methods and technologies for recognizing linguistic patterns in the textual data. The main goal of this axis of research is the development of a computational historiography that allows researchers to recon-struct the history and geography of artistic and art-historical concepts and ideas, by being able to trace art historical terminology and argumentation contextually. Indeed, scholarly experience shows that studying the history of Art History becomes increasingly challenging after 1900 due to the large quantity of textual material. On the other hand, since the 1990s, textual scholarship itself has become digital. In this light, digital research methods become not only necessary (due to the scale and complexity of the material) but contextually appropriate (with increasingly born-digital scientific publications) to map understand the development of the intellectual history and geography of the discipline in the last century.
This shift towards a corpus historiography echoes previous similar shifts in corpus linguistics: such a historiography, however, requires different computational methods to those developed for linguistic and literary phenomena. Novel computational problems include multilingualism (the interrelation between language-groups and schools of scientific thought), contextualization (particularly in relation to intertextual and intervisual references), and polysemy, which however has well progressed over the last two decades thanks to the Getty Research Institute and others, including the BHMPI. New textual search engines will change the virtual structure of the BHMPI library and its scholarly use significantly, whilst the semantic analysis of text can lead towards a recommendation engine for secondary references for links made in the BHMPI’s knowledge-graph. Specific tools for identifying handwriting will be necessary for the identification of the digitized photo material.
The spatiotemporal domain not only allows visual data to be linked with the textual data (interconnecting, for instance, the BHMPI’s library and photographic collection), but also enables new forms of organizing and representing these data through historical space and time. The rapidly- expanding field of the Digital Spatial Humanities will be particularly relevant to this research axis, using a cross-scalar approach: from micro-resolution 3D scanning of artistic artefacts and architectural details, to urban- and continent- scale geo-mapping. Recent advances in differentiable rendering systems, and a more general convergence between computer graphics and vision methods, allow for 3D data to be not only reproduced but analyzed. Methods for the simulation and analysis of three-dimensional artefacts and architectural spaces will be developed, building on current work in acoustic simulation, light simulation and space-syntax analysis. Measures of centrality and accessibility analysis can add invaluable insights about the how the spatial material conditions of a rich and cumulative urban tissue (such as Rome) influences the material practices through access, expo-sure and co-presence of resources, of artists, of galleries, of publics, of institutions, etc. As well as adapting current methods from architecture, computational urban studies, spatial econometrics and acoustic modelling, novel techniques must be developed which take account of the important temporal dimension of historic data (so-called 4D modelling), as well as the unique uncertainties in simulation and modelling that arise from philological and archaeological evidence.
Addressing the visual can hardly be done without addressing other data modalities such as text. The collections of the BHMPI we encourage our fellows to tackle require interconnecting different assets and types of data. In this respect, Multimodal ML is among the newest and most promising fields in Machine Learning and can be used to model complex and high-level concepts, like 'style', which make for the most challenging and interesting research topics in the humanities.