TACOS: Temporally-aligned Audio CaptiOnS for Language-Audio Pretraining

TACOS is a collection of 12,358 audio recordings, annotated with 47,748 temporally strong audio captions (i.e., textual descriptions of sound events and their corresponding temporal onsets and offsets). Each audio file is additionally paired with a weak caption, which was automatically generated from the strong captions using OpenAI’s gpt-4o-mini-2024-07-18.

If you use TACOS in your work, please cite our paper (preprint available on arXiv).

A usage example, and more deatils are provided in our GitHub repository (available soon).

Licensing

The audio recordings contained in audio.zip and the associated metadata in metadata.csv were sourced from the FreeSound platform [1]. Each recording and its corresponding metadata are governed by their individual licenses, which are specified—along with the creator's information—in the metadata.csv file.

The captions provided in annotations_strong.csv and annotations_weak.csv are licensed under the CC BY 4.0, which permits use under the condition of appropriate attribution.

References

[1] Frederic Font, Gerard Roma, and Xavier Serra. Freesound technical demo. In Proceedings of the 21st ACM Multimedia Conference (MM '13), Barcelona, Spain, October 21-25, 2013