DESCRIPTION:
\n\nThe Sony-TAu Realistic Spatial Soundscapes 2023 (STARSS23) dataset contains multichannel recordings of sound scenes in various rooms and environments, together with temporal and spatial annotations of prominent events belonging to a set of target classes. The dataset is collected in two different countries, in Tampere, Finland by the Audio Researh Group (ARG) of Tampere University (TAU), and in Tokyo, Japan by SONY, using a similar setup and annotation procedure. The dataset is delivered in two 4-channel spatial recording formats, a microphone array one (MIC), and first-order Ambisonics one (FOA). These recordings serve as the development dataset for the DCASE 2023 Sound Event Localization and Detection Task of the DCASE 2023 Challenge.
\n
\nThe STARSS23 dataset is a continuation of the STARSS22 dataset. It extends the previous version with the following:
Contrary to the three previous datasets of synthetic spatial sound scenes of TAU Spatial Sound Events 2019 (development/evaluation), TAU-NIGENS Spatial Sound Events 2020, and TAU-NIGENS Spatial Sound Events 2021 associated with previous iterations of the DCASE Challenge, the STARS22-23 dataset contains recordings of real sound scenes and hence it avoids some of the pitfalls of synthetic generation of scenes. Some such key properties are:
\n\nThe first round of recordings was collected between September 2021 and January 2022. A second round of recordings was collected between November 2022 and February 2023.
\n
\nCollection of data from the TAU side has received funding from Google.
REPORT & REFERENCE:
\n\nIf you use this dataset you could cite this report on its design, capturing, and annotation process:
\n\nArchontis Politis, Kazuki Shimada, Parthasaarathy Sudarsanam, Sharath Adavanne, Daniel Krause, Yuichiro Koyama, Naoya Takahashi, Shusuke Takahashi, Yuki Mitsufuji, Tuomas Virtanen (2022). STARSS22: A dataset of spatial recordings of real scenes with spatiotemporal annotations of sound events. In Proceedings of the Detection and Classification of Acoustic Scenes and Events 2022 Workshop (DCASE2022), Nancy, France.
\n\nfound here.
\n\nA more detailed report on the properties of the new dataset and its audiovisual processing with a suitable baseline for DCASE2023 will be published soon.
\n\nAIM:
\n\nThe STARSS22-23 dataset is suitable for training and evaluation of machine-listening models for sound event detection (SED), general sound source localization with diverse sounds or signal-of-interest localization, and joint sound-event-localization-and-detection (SELD). Additionally, the dataset can be used for evaluation of signal processing methods that do not necessarily rely on training, such as acoustic source localization methods and multiple-source acoustic tracking. The dataset allows evaluation of the performance and robustness of the aforementioned applications for diverse types of sounds, and under diverse acoustic conditions.
\n\nSpecifically the STARSS23 allows additionally evaluation of audiovisual processing methods, such as audiovisual source localization.
\n\nSPECIFICATIONS:
\n\nGeneral:
\n\nVolume, duration, and data split:
\n\nAudio:
\n\nVideo:
\n\nMore detailed information on the dataset can be found in the included README file.
\n\nSOUND CLASSES:
\n\n13 target sound event classes are annotated. The classes follow loosely the Audioset ontology.
\n\n 0. Female speech, woman speaking
\n 1. Male speech, man speaking
\n 2. Clapping
\n 3. Telephone
\n 4. Laughter
\n 5. Domestic sounds
\n 6. Walk, footsteps
\n 7. Door, open or close
\n 8. Music
\n 9. Musical instrument
\n 10. Water tap, faucet
\n 11. Bell
\n 12. Knock
The content of some of these classes corresponds to events of a limited range of Audioset-related subclasses. For more information see the README file.
\n\nEXAMPLE APPLICATION:
\n\nAn implementation of a trainable model of a convolutional recurrent neural network, performing joint SELD, trained and evaluated with this dataset is provided here. This implementation will serve as the baseline method for the audio-only track in the DCASE 2023 Sound Event Localization and Detection Task.
\n
\nA baseline for the audiovisual track of DCASE 2023 Sound Event Localization and Detection Task will be published soon and referenced here.
DEVELOPMENT AND EVALUATION:
\n\nThe current version (Version 1.0) of the dataset includes only the 168 development audio/video recordings and labels, used by the participants of Task 3 of DCASE2023 Challenge to train and validate their submitted systems. Version 1.1 will be including additionally the evaluation audio and video recordings without labels, for the evaluation phase of DCASE2023.
\n\nIf researchers wish to compare their system against the submissions of DCASE2023 Challenge, they will have directly comparable results if they use the evaluation data as their testing set.
\n\nDOWNLOAD INSTRUCTIONS:
\n\nThe file foa_dev.zip, correspond to audio data of the FOA recording format.
\nThe file mic_dev.zip, correspond to audio data of the MIC recording format.
The file video_dev.zip contains the common videos for both audio formats.
\nThe file metadata_dev.zip contains the common metadata for both audio formats.
Download the zip files corresponding to the format of interest and use your favourite compression tool to unzip these zip files.
", "publication_date": "2023-03-08", "publisher": "Zenodo", "references": [ { "reference": "Archontis Politis, Kazuki Shimada, Parthasaarathy Sudarsanam, Sharath Adavanne, Daniel Krause, Yuichiro Koyama, Naoya Takahashi, Shusuke Takahashi, Yuki Mitsufuji, Tuomas Virtanen (2022). STARSS22: A dataset of spatial recordings of real scenes with spatiotemporal annotations of sound events. In Proceedings of the Detection and Classification of Acoustic Scenes and Events 2022 Workshop (DCASE2022), Nancy, France." }, { "reference": "Archontis Politis, Sharath Adavanne, Daniel Krause, Antoine Deleforge, Prerak Srivastava, Tuomas Virtanen (2021). A Dataset of Dynamic Reverberant Sound Scenes with Directional Interferers for Sound Event Localization and Detection.\u00a0 In Proceedings of the Detection and Classification of Acoustic Scenes and Events 2021 Workshop (DCASE2021), Barcelona, Spain." }, { "reference": "Archontis Politis, Sharath Adavanne, and Tuomas Virtanen (2020). A Dataset of Reverberant Spatial Sound Scenes with Moving Sources for Sound Event Localization and Detection. In Proceedings of the Detection and Classification of Acoustic Scenes and Events 2020 Workshop (DCASE2020), Tokyo, Japan." }, { "reference": "Sharath Adavanne, Archontis Politis, and Tuomas Virtanen (2019). A Multi-room reverberant dataset for sound event localization and detection. Proceedings of the Detection and Classification of Acoustic Scenes and Events 2019 Workshop (DCASE2019), New York, NY, USA." } ], "related_identifiers": [ { "identifier": "10.5281/zenodo.6600531", "relation_type": { "id": "isnewversionof", "title": { "de": "Ist eine neue Version von", "en": "Is new version of" } }, "resource_type": { "id": "dataset", "title": { "de": "Datensatz", "en": "Dataset" } }, "scheme": "doi" }, { "identifier": "10.5281/zenodo.5476980", "relation_type": { "id": "references", "title": { "de": "Referenziert", "en": "References" } }, "resource_type": { "id": "dataset", "title": { "de": "Datensatz", "en": "Dataset" } }, "scheme": "doi" }, { "identifier": "10.5281/zenodo.4064792", "relation_type": { "id": "references", "title": { "de": "Referenziert", "en": "References" } }, "resource_type": { "id": "dataset", "title": { "de": "Datensatz", "en": "Dataset" } }, "scheme": "doi" }, { "identifier": "10.5281/zenodo.2599196", "relation_type": { "id": "references", "title": { "de": "Referenziert", "en": "References" } }, "resource_type": { "id": "dataset", "title": { "de": "Datensatz", "en": "Dataset" } }, "scheme": "doi" }, { "identifier": "10.5281/zenodo.3377088", "relation_type": { "id": "references", "title": { "de": "Referenziert", "en": "References" } }, "resource_type": { "id": "dataset", "title": { "de": "Datensatz", "en": "Dataset" } }, "scheme": "doi" }, { "identifier": "https://github.com/sharathadavanne/seld-dcase2022", "relation_type": { "id": "references", "title": { "de": "Referenziert", "en": "References" } }, "resource_type": { "id": "software", "title": { "de": "Software", "en": "Software" } }, "scheme": "url" } ], "resource_type": { "id": "dataset", "title": { "de": "Datensatz", "en": "Dataset" } }, "rights": [ { "description": { "en": "A short and simple permissive license with conditions only requiring preservation of copyright and license notices. Licensed works, modifications, and larger works may be distributed under different terms and without source code." }, "id": "mit", "props": { "scheme": "spdx", "url": "https://opensource.org/licenses/MIT" }, "title": { "en": "MIT License" } } ], "subjects": [ { "subject": "sound event detection" }, { "subject": "sound source localization" }, { "subject": "spatial audio" }, { "subject": "360-degree video" }, { "subject": "audiovisual processing" }, { "subject": "acoustic scene analysis" }, { "subject": "source distance estimation" }, { "subject": "machine listening" }, { "subject": "Ambisonics" } ], "title": "STARSS23: Sony-TAu Realistic Spatial Soundscapes 2023", "version": "1.0.0" }, "parent": { "access": { "owned_by": { "user": "96758" } }, "communities": { "default": "2d7be671-0ce3-4c1e-9ed3-cfc92571ad1a", "entries": [ { "access": { "member_policy": "open", "members_visibility": "public", "record_submission_policy": "open", "review_policy": "open", "visibility": "public" }, "children": { "allow": false }, "created": "2019-11-25T09:04:41.826171+00:00", "custom_fields": {}, "deletion_status": { "is_deleted": false, "status": "P" }, "id": "2d7be671-0ce3-4c1e-9ed3-cfc92571ad1a", "links": {}, "metadata": { "curation_policy": "", "description": "Datasets related to the DCASE challenges.", "page": "The goal of this communty is to gather datasets related to the DCASE challenge series. More information available on DCASE website.
\r\n", "title": "Detection and Classification of Acoustic Scenes and Events" }, "revision_id": 0, "slug": "dcase", "updated": "2019-11-25T09:06:23.594750+00:00" } ], "ids": [ "2d7be671-0ce3-4c1e-9ed3-cfc92571ad1a" ] }, "id": "7709051", "pids": { "doi": { "client": "datacite", "identifier": "10.5281/zenodo.7709051", "provider": "datacite" } } }, "pids": { "doi": { "client": "datacite", "identifier": "10.5281/zenodo.7709052", "provider": "datacite" }, "oai": { "identifier": "oai:zenodo.org:7709052", "provider": "oai" } }, "revision_id": 7, "stats": { "all_versions": { "data_volume": 74403649423428.0, "downloads": 32220, "unique_downloads": 18218, "unique_views": 12066, "views": 13514 }, "this_version": { "data_volume": 12841548922373.0, "downloads": 5639, "unique_downloads": 3596, "unique_views": 2851, "views": 3220 } }, "status": "published", "swh": {}, "updated": "2023-06-26T13:33:06.256826+00:00", "versions": { "index": 1, "is_latest": false } }