Candidate-Affiliation Dataset, Code, and Supplementary Information

Overview

This record accompanies the paper Political Stance Detection on X During the 2024 U.S. Election: A Comparative Study of Classical, Neural, and Transformer Models. It extends the USE24-XD dataset (10.5281/zenodo.22803534) and contains:

• Dataset: X (Twitter) posts labeled with political candidate affiliation (Trump, Biden_Kamala, Neutral), plus engineered features. The features cover sentiment, hate and misinformation signals, temporal context, engagement, location, and LDA topic tokens.
• Human evaluation set: 300 posts labeled by two annotators.
• Code: notebooks for the stance-detection experiments (classical, neural and transformer models) and the run-to-run variance analysis.
• Supplementary Information (SI): the paper's SI as a PDF, covering model configurations and cross-validation robustness.

Intended uses:

• Political discourse analysis
• Temporal dynamics of narratives
• Stance detection, topic modeling and other NLP tasks

Citation

If you use this dataset or build on it in your research, please cite:

TBD

Please also cite the source dataset, USE24-XD (10.5281/zenodo.22803534).

Post text and access

This record does not contain raw post text, usernames or author IDs. Each post is identified only by its X post id. The only text-derived content is lda_tokens: lemmatized, stop-word-filtered tokens with no word order.

To get the original post text:

• Full text and metrics: request access through the USE24-XD record, 10.5281/zenodo.22803534.
• Dehydrated IDs (public): 10.5281/zenodo.22777827. Rehydrate them through the X API.

Join on id in both cases. Read id as a string, because it exceeds 64-bit float precision.

Contents

Data schema: annotation_dataset.csv

Each row is one post. Posts span 2024-10-17 to 2025-07-16.

Label distribution

Notes

• Labeling. candidate_affiliation comes from a keyword-and-stance heuristic. Candidate-skewed vocabulary was refined with log-odds ratios and then combined with stance cues. See the "Candidate fan heuristic" section of stance_detection_experiment.ipynb, where this column is named candidate_fan.

• Temporal period. temporal_period is used as a feature in the experiments. It is not stored in the file; derive it from tweet_date as below. Posts dated 2024-10-17, before the first boundary, have no period.

  temporal_period	Dates (UTC)
  election_period	2024-10-18 – 2024-11-04
  election_day	2024-11-05
  post_election	2024-11-06 – 2025-07-16

• Content warning. lda_tokens contains offensive language, because it reflects the original posts.

Sample rows

Loading

Human evaluation: human_evaluation.csv

300 posts were sampled from the dataset. Two annotators labeled each post independently, and disagreements were resolved into a final label. Post text is not included; join on id with USE24-XD to view it.

• Inter-annotator agreement: 87.3% raw, Cohen's κ = 0.81
• Agreement between the heuristic candidate_affiliation and final_label: 89.7% (269 / 300)

Code

stance_detection_experiment.ipynb

This notebook implements the full pipeline. All models share the same preprocessing and evaluation setup.

1. Data preprocessing

• English filter, text normalization and cleaning
• Removal of URLs, mentions and noise
• Tokenization and lemmatization
• Handling of hashtags and emoji
• Removal of short or empty posts, and de-duplication

2. Feature engineering

• Text: TF-IDF (unigrams + bigrams) and SBERT sentence embeddings (all-mpnet-base-v2)
• Sentiment: score in [-1, 1] and label
• Hate and misinformation signals: keyword-based counts and binary flags
• Temporal: hour of day, day of week, event-based temporal period
• Engagement: raw count, normalized rate, log-transformed rate
• Metadata: state-level user location, platform sensitivity flag
• Topic modeling: LDA tokens

3. Models

• Classical:
  • Logistic regression (TF-IDF)
  • Linear SVM
  • Histogram gradient boosting (HGB with SVD)
• Neural: SBERT (all-mpnet-base-v2) embeddings + MLP classifier
• Transformer: BERTweet (vinai/bertweet-base), fine-tuned

4. Training setup

• Stratified split: 64% train, 16% validation, 20% test
• Training classes balanced by upsampling
• Early stopping on validation macro-F1
• Hyperparameter tuning for all models

5. Evaluation

• Macro-F1 (primary metric)
• Accuracy
• Confusion matrices for error analysis

mlp_variance_analysis.ipynb

This notebook measures the run-to-run variability of the SBERT + MLP model. It retrains the model under three protocols:

1. 10 random seeds on the fixed 64/16/20 split
2. Stratified 5-fold cross-validation
3. 2,000-sample bootstrap 95% CIs on the test set

Each feature configuration is compared with the text-only baseline using paired tests with Holm correction. When run, the notebook writes per-run metrics, summary tables and a macro-F1 figure.

MLP settings:

• Hidden layers [256, 128], dropout 0.5
• AdamW (lr 5e-4, weight decay 1e-3)
• Label smoothing 0.1, batch size 128
• Up to 40 epochs, early-stopping patience 7

Feature configurations:

Supplementary Information: supplementary_information.pdf

The paper's Supplementary Information (6 pages):

• Appendix A — Model configurations. Original baseline vs. final settings for each model family:
  • Table A1: logistic regression and linear SVM
  • Table A2: MLP variants
  • Table A3: BERTweet
• Appendix B — Cross-validation robustness. The feature analyses from the main text, repeated with stratified 5-fold cross-validation (64/16/20 within each fold) and the Nadeau–Bengio corrected resampled t-test with Holm correction:
  • Table B4: incremental feature blocks (P, A–D)
  • Table B5: single-feature ablation

How to run

The notebooks start from a full table that includes post text (cleaned_election_tweets.csv). To rebuild it, join this dataset with the USE24-XD post text on id (see Post text and access). Notebook outputs that displayed post text were cleared before release.

Ethics and intended use

• For research on political discourse, stance detection and NLP methods only.
• Do not use it to identify, profile or target individual users.
• Location is coarse: it is inferred at the state level from free-text profiles and is often unreliable.
• Hate and misinformation flags come from simple keyword lists. They are not validated classifiers.
• Anyone using rehydrated content must follow the X Developer Agreement and Policy.