Cathedral, bazaar, data garden: the Pangloss Collection

Presented by Alexis Michaud and Séverine Guillaume at the Language Documentation and Archiving Conference, Berlin & Online, 4-6 Sept, 2024

Eric Raymond’s classic The Cathedral and the Bazaar (Raymond 1999) contrasts top-down and bottom-up design. How does the Pangloss Collection (pangloss.cnrs.fr), a member of the Digital Endangered Languages and Musics Archives Network, pattern in these terms? There is a cathedral aspect to the format used for display and distribution: hierarchical markup (XML) following a fixed structure that has remained essentially stable over the years (Jacobson, Michailovsky & Lowe 2001; Michailovsky et al. 2014). Yet the collection is on the bazaar side in terms of the corpora hosted. Any speech dataset can in principle be deposited in the archive, provided that it was collected in connection to linguistic/anthropological research (and belongs within the national scope, since the institution taking care of long-term archiving has national scope). Corpora are archived as is: they are documented to a varying extent, they have vastly different sizes, some are annotated with an excruciating amount of detail whereas others only have a transcription and some are untranscribed, or even left untranscribed. The diversity of the archive, and the evolution of the hosted resources over time, suggest a third metaphor: that of the data garden, which not only undergoes overall growth over the years, but also allows (and fosters) gradual improvements to the resources. The gardening tasks can favour emulation and cross-fertilization in terms of annotation practices, without hard regulations and guidelines. The presentation of the archive will place emphasis (i) on the functions that it plays as part of a broader Open Science environment, (ii) on current plans for improvements to workflows, to interfaces, and to resources and metadata, and (iii) on opportunities, challenges and threats related to Natural Language Processing (such as automatic transcription: Guillaume et al. 2022).