Parasitic pixels – How cameras (and their settings) can jeopardise image-based 3D modelling

Description

Abstract (English)

Continuous and discrete digital representations of a surface can either be constructed from scratch in a Geographic Information System, Computer-Aided Design or Computer Graphics environment, or obtained through digitising the surface geometry of an existing physical object or scene. Amongst the many application-driven techniques developed by multiple disciplines, one possible approach to surface digitisation is via photographs (or more general images), commonly referred to as Image-Based (3D) (surface) Modelling or simply IBM.

IBM encompasses different techniques, but most rely on object or scene photographs taken from different locations to extract digital surface data. IBM has been the focus of both the photogrammetric and computer vision fields. However, hybrid photogrammetric computer vision-based approaches like Structure from Motion (SfM), Simultaneous Location/Localisation And Mapping (SLAM), and Multi-View Stereo (MVS) have become commonplace in cultural heritage documentation over the past 1.5 decades. This rapid technology uptake also led to an ever-increasing and continuous stream of new literature, filled almost exclusively with software comparisons or developments towards faster and more automated approaches.

However, one thing stands out in this story: the surprisingly little attention devoted to the input of all these algorithms: the photographs. Are digital cameras so ubiquitous, and have algorithms become so robust that academics stopped paying attention (if they ever really did?) to how a camera’s operating principles tailor the pixels they generate?

This talk will go back to the basics of all IBM approaches – pixels – and detail how camera architectures and settings affect their positional and spectral values. Various case studies will illustrate how the resulting sub-optimal or parasitic pixels can negatively affect the IBM output and which measures can mitigate such issues. Ultimately, the paper hopes to incentivise more awareness for, and academic research on, the basics of cultural heritage IBM: photography.