Face2Text data set readme

-----
Version
-----
2.1

-----
Release date
-----
15/04/2024

-----
Contents
-----
readme.txt: Information about this data set.
raw_2.1.json: Data set with all descriptions included and left unaltered as written by annotators (version 2.0 removed whitespace from the edges of descriptions but here we leave them actually unaltered).
clean_train_2.1.json: 80% train split from cleaned up data set (12,918 descriptions / 8,489 images).
clean_dev_2.1.json: 10% dev split from cleaned up data set (2,887 descriptions / 1,869 images).
clean_test_2.1.json: 10% test split from cleaned up data set (2,885 descriptions / 1,869 images).
cleaner: A folder containing the script that converted the raw data into the cleaned and split data.

-----
Paper
-----
URL: https://aclanthology.org/2022.pvlam-1.6/
Authors: Tanti, Marc and Abdilla, Shaun and Muscat, Adrian and Borg, Claudia and Farrugia, Reuben A. and Gatt, Albert
Year: 2022
Title: Face2Text revisited: Improved data set and baseline results

-----
Stats
-----
Number of images: 12,227
Number of descriptions: 18,690
Maximum number of descriptions per image: 5 (0.1%)
Minimum number of descriptions per image: 1 (61.4%)

-----
Description
-----
Face2Text is an ongoing project to collect a data set of natural language descriptions of human faces. A randomly selected sample of images was taken from the CelebA data set (https://www.kaggle.com/jessicali9530/celeba-dataset) and was presented to several human annotators who each had a random sample of faces to describe.

The raw data set is provided in JSON format structured as a list of objects consisiting of the following:
    response_id: A unique ID referring to a unique annotator/image pair.
    filename: The file name of the image in the CelebA data set.
    user_id: A unique ID referring to the annotator that wrote the description.
    description: The description written.

Here is an example:
    [
        {
            "response_id": "4683",
            "filename": "000035.jpg",
            "user_id": "2", //Not in the cleaned data.
            "description": "A woman with a chiselled jaw, prominent cheekbones, a long, narrow nose and thin eyebrows. She has long, messy, black hair and she is wearing makeup."
        },
        ...
    ]

The images file name refers to a face photo in the CelebA data set (https://www.kaggle.com/jessicali9530/celeba-dataset#img_align_celeba.zip). We do not include images here; you will need to download them and select them separately.

Instructions given to annotators were as follows:
    @ Describe these faces as naturally as possible.
    @ Do not spend too much time thinking about what to write. Just write the description which, in your view, accurately captures the physical attributes of the face.
    @ Don't describe the background and don't make inferences about the situation of the photo or the person (such as the person's job or background).
    @ You can describe a person's facial expression or their emotional state if this is evident from the picture.

The raw data set is useful to analyse how people write descriptions of faces and the user ID is provided to distinguish between different annotators. The cleaned data set is useful for machine learning tasks as it was cleaned in order to reduce the amount of noise and typos in the raw data set, of which there is a substantial amount. Given that there was an effort made to standardise the descriptions across authors, the user_id column was dropped from the clean data, but the response_id was left so that it is still possible to link a cleaned description to its original raw version. Descriptions of the same image were all kept within the same split to avoid data leakage.

Cleaning was done using a series of text replacements that are documented in a script which is included with the data set. These replacements correct typos and remove noise (there is evidence of automatic speech recognition being used by some annotators such as the phrase "pear of" instead of "pair of"), replace references to countries such as "An Indian looking man" with more generic references such as "A south Asian looking man", remove redundant use of hyphenation such as "clean-shave", fix capitalisation and punctuation, and split long sentences into smaller ones. Although the replacement rules were made manually after some manual inspection (all changes made to the data set are almost guaranteed to be improvements), there was no thorough manual inspection of the entire data set carried out in this version of the data set. There are also known potential further improvements which were not carried out in this version, such as removing redundant phrases like "This is a photo of...". A future version will hopefully include a thorough scrubbing of all the descriptions with more detail added to short descriptions.

-----
License
-----
The MIT License (MIT)

Copyright © 2024 Marc Tanti, Shaun Abdilla, Adrian Muscat, Claudia Borg, Reuben A. Farrugia, Albert Gatt

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
