Making Machine Learning Worlds

Darío Negueruela and Iacopo Neri present at the Royal Geographical Society’s 2023 Annual International Conference held at the Imperial College London and Royal Geographical Society, London, UK, August 31-September 1 2023

They present their paper entitled Urban Gist in the framework of the session ‘Making Machine Learning Worlds’ (from the European Research Council funded-project ’Algorithmic Societies: Ethical Life in the Machine Learning Age’), convened by Louise Amoore and Ludovico Rella from Durham University.

Abstract

We discuss a computational pipeline for investigating the cultural landscape of a city through the eyes of a machine, and for questioning modes of embedding culture in machine learning models, while being open towards novel research lines for social studies. By studying the outputs of machine learning models extracting features and textual properties from images, we aim to reveal underlying relationships between the history and architecture of a city and its urban development, finally adopting Rome as a pivoting case study.

Technically, by feeding 360° panoramic images into large vision-language models (e.g. diffusion models, or contrastive learning based like CLIP or ALIGN), we question how mainstream culture is expressed in these models, and wonder how art and technology can intertwine to originally render a city. In this machine-triggered urban experiment, we investigate overlaps between history and machinic interpretation and whether relevant temporal correlations can be captured through generic images only. Finally, by spatially analyzing the captured data, we identify clusters and discontinuities in the urban layout that visually depict the interplay of forces behind its development.

As in a forensic exercise, the paper seeks to uncover the complex social and historical dynamics of urban environments, exploiting only contemporary images of their settings and a generic embedding of culture. It explores potential cultural biases embedded in machine learning models by comparing Rome – culturally relevant for the western world – with other cities around the world; leveraging innovative computational pipelines and globally covering datasets to provide a novel research line for urban studies.