The language of sound search: Examining User Queries in Audio Search Engines (supplementary materials)

Overview

This dataset accompanies the paper titled "The Language of Sound Search: Examining User Queries in Audio Search Engines." The study investigates user-generated textual queries within the context of sound search engines, which are commonly used for applications such as foley, sound effects, and general audio retrieval.

The paper addresses the gap in current research regarding the real-world needs and behaviors of users when designing text-based audio retrieval systems. By analyzing search queries collected from two sources — a custom survey and Freesound query logs — the study provides insights into user behavior in sound search contexts. Our findings reveal that users tend to formulate longer and more detailed queries when not constrained by existing systems, and that both survey and Freesound queries are predominantly keyword-based.

This dataset contains the raw data collected from the survey and annotations of Freesound query logs.

Files in This Dataset

The dataset includes the following files:

1. participants.csv
   Contains data from the survey participants. Columns:

   • id: A unique identifier for each participant.
   • fluency: Self-reported English language proficiency.
   • experience: Whether the participant has used online sound libraries before.
   • passed_instructions: Boolean value indicating whether the participant advanced past the instructions page in the survey.

2. annotations.csv
   Contains annotations of the survey responses, detailing the participants' interaction with the sound search tasks. Columns:

   • id: A unique identifier for each annotation.
   • participant_id: Links to the participant’s ID in participants.csv.
   • stimulus_id: Identifier for the stimulus presented to the participant (audio, image, or text description).
   • stimulus_type: The type of stimulus (audio, image, text).
   • audio_result_id: Identifier for the hypothetical audio result presented during the search task.
   • query1: Initial search query submitted based on the stimulus.
   • query2: Refined search query after seeing the hypothetical search result.
   • aspects1: Aspects considered important when formulating the initial query.
   • aspects2: Aspects considered important when refining the query.
   • result_relevance: Participant's rating of the hypothetical search result's relevance.
   • time: Time taken to complete the search task.

3. freesound_queries_annotated.csv
   Contains annotated Freesound search queries. Columns:

   • query: Text of the search query submitted to Freesound.
   • count: The number of times the specific query was submitted.
   • topic: Annotated topic of the query, based on an ontology derived from AudioSet, with an additional category, Other, which includes non-English queries and NSFW-related content.

4. survey_stimuli_data.zip
   This ZIP file contains three CSV files corresponding to the three stimulus types used in the survey:

   • Audio stimuli: Categorized sound recordings presented to participants.
   • Image stimuli: Annotated images that prompted sound-related queries.
   • Text stimuli: Summarized descriptions of sounds provided to participants.

More details on the stimuli and the survey methodology can be found in the accompanying paper.

Citation

If you use this dataset in your research, please cite the corresponding paper:

B. Weck and F. Font, ‘The Language of Sound Search: Examining User Queries in Audio Search Engines’, in Proceedings of the Detection and Classification of Acoustic Scenes and Events 2024 Workshop (DCASE2024), Tokyo, Japan, Oct. 2024, pp. 181–185.

@inproceedings{Weck2024,
    author = "Weck, Benno and Font, Frederic",
    title = "The Language of Sound Search: Examining User Queries in Audio Search Engines",
    booktitle = "Proceedings of the Detection and Classification of Acoustic Scenes and Events 2024 Workshop (DCASE2024)",
    address = "Tokyo, Japan",
    month = "October",
    year = "2024",
    pages = "181--185"
}