BenSyc: Benchmarking Conversational Sycophancy and Human Alignment in LLMs for Bengali Contexts

BenSyc is a culturally grounded benchmark for studying conversational sycophancy, social reinforcement, and alignment behavior in Bengali and Banglish online interactions.

The benchmark consists of manually validated Reddit post–comment pairs collected from Bengali-speaking online communities spanning Bangladesh and West Bengal. The dataset preserves naturally occurring Bengali, Banglish, English code-switching, emojis, slang, and informal social-media language to support realistic multilingual conversational evaluation.

BenSyc provides:
- Binary conversational alignment labels
- Fine-grained five-class conversational alignment annotations:
  Invalidation, Neutral, Support, Validation, and Escalation
- Human-validated rationales and evidence spans
- Prompt templates used for evaluation
- Evaluation scripts and benchmarking utilities

The benchmark was designed to evaluate how large language models respond to emotionally sensitive social conversations, including advice-seeking, interpersonal conflict, emotional support, and socially grounded reasoning.

The dataset supports:
- Conversational sycophancy detection
- Fine-grained conversational alignment classification
- Response generation evaluation
- Multilingual conversational safety research
- Cultural alignment analysis for LLMs

To protect user privacy and support responsible research release practices, personally identifying metadata, usernames, URLs, timestamps, and Reddit identifiers were removed during preprocessing.

This repository accompanies the paper:

“BenSyc: Benchmarking Conversational Sycophancy and Human Alignment in LLMs for Bengali Contexts”

License:
CC BY-NC 4.0

Intended Use:
This dataset is intended strictly for academic research and evaluation purposes.