Data and code for project "What makes users click: the effect of news values negativity and surprise in news headlines on the CTR"

This repository includes the following files and corresponding Python codes saved in Jupiter Notebooks:

upworthy-archive-confirmatory-packages-03.12.2020.csv: Original dataset downloaded from The Upworthy Research Archive (https://osf.io/jd64p/)

cleaned_data.csv: upworthy-archive-confirmatory-packages-03.12.2020.csv cleaned with notebook_clean_data.ipynb

• Data sorted by clickability_test_id
• Headlines with 0 clicks deleted
• Variables ctr, mean_ctr, and lift created
• Variable winner converted to dummy variable
• Only kept first occurrence of headlines that occur multiple times in the dataset per test
• Deleted tests with only one headline

sample_manual_annotations.csv: sample of 300 headlines of cleaned_data.csv retrieved with notebook_sample_manual_annotations.ipynb

inter-annotator_sentiment.csv: input for notebook_inter-annotator_scores_confusion_matrix.ipynb to calculate inter-annotator agreement scores with regard to annotating sentiment

inter-annotator_emotion.csv: input for notebook_inter-annotator_scores_confusion_matrix.ipynb to calculate inter-annotator agreement scores with regard to annotating emotion

input_sentiment.tsv: input for notebook_sentiment_emotion_test_set.ipynb for sentiment analysis on sample of 300 headlines in sample_manual_annotations.csv

input_emotion.tsv: input for notebook_sentiment_emotion_test_set.ipynb for emotion analysis on sample of 300 headlines in sample_manual_annotations.csv

input_all_headlines.tsv: input for notebook_sentiment_emotion_final_annotation.ipynb to annotate all headlines in cleaned_data.csv

final_dataset.csv: output from notebook_merge_dataframes.ipynb that merges all csv output files  from notebook_sentiment_emotion_final_annotation.ipynb with cleaned_data.csv

notebook_descriptive_statistics.ipynb: Python code to retrieve descriptive statistics of various variables in cleaned_data.csv

NRC-Emotion-Lexicon-Wordlevel-v0.92.txt: NRC Emotion Lexicon used in notebook_sentiment_emotion_test_set.ipynb and notebook_sentiment_emotion_final_annotation.ipynb