TonalityPrint: A Contrast-Structured Voice Dataset for Exploring Functional Tonal Intent, Ambivalence, and Inference-Time Prosodic Alignment v1.0

Description

Abstract

TonalityPrint is a specialized, controlled, single-speaker prosody dataset designed for precision-tuning functional tonal intents that govern complex human conversation in voice AI systems. The corpus provides 144 human-verified audio samples across 18 utterances, each recorded in 8 parallel prosodic states: Baseline/Neutral, five core Functional Tonal Intents (Trust, Attention, Reciprocity, Empathy Resonance, Cognitive Energy)(Picard, 1997; Cutler et al., 1997), and systematically annotated Ambivalence states, treating tonal complexity as a learnable feature rather than annotation error (Cowen & Keltner, 2017; Pell & Kotz, 2011) .

Core Innovation: TonalityPrint proposes hypothesized data-level substrates that may enable prosodic AI alignment:

1. Ambivalence Annotation: Unlike traditional emotion datasets that treat mixed or transitional signals as noise, TonalityPrint systematically annotates ambivalence as a perceptual entropy cross-intent feature, aiming to provide an operational reference signal for AI systems that must navigate real-world tonal complexity during inference.

2. Differential Latent Analysis (DLA): Maintaining speaker identity and lexical content constant across parallel prosodic states (the "Fixed-Phrase Octet"), TonalityPrint may enable researchers to perform contrastive approximation of tonal intent vectors - contrastive approximation of tonal intent vectors, analogous to established activation-steering methods in LLMs but applied to voice prosody (Rimsky et al., 2024; Anthropic, 2024). These represent proposed methodological extensions and hypothesis-generating proposals requiring empirical validation at scale.

Empirical Context: Annotations are grounded in ecological feasibility derived from 8,873+ consequential customer interactions, where the observed tonal patterns correlated with a stable ~35.85% average conversion rate (ecological provenance rather than proof).  During this period, 68 listeners (~0.76% of interactions) spontaneously described the speaker's voice using AI-adjacent descriptors ('automated,' 'TV voice') while maintaining high trust and positive outcomes. While these observations are confounded by numerous variables and cannot establish causation, they motivated the hypothesis that certain prosodic patterns merit systematic investigation for human-AI voice alignment - particularly the counterintuitive finding that "AI-adjacent" vocal qualities may co-occur with trust-building (Breazeal, 2003) rather than necessarily triggering discomfort typically associated with the well-documented ‘uncanny valley’ effects.

Theoretical Foundation: These innovations operationalize the Tonality as Attention framework (Polhill, 2025), which proposes that human vocal prosody serves as an active attention mechanism functionally analogous to computational attention mechanisms in AI architectures - a shared signaling system that may bridge human-machine communication. TonalityPrint aims to provide an early controlled dataset for testing this framework's core hypothesis: that prosodic patterns function as steerable intent vectors during inference.

Dataset Structure: The corpus provides exploratory continuous indices (0-100) for Trust, Attention, Reciprocity, Empathy Resonance, and Cognitive Energy intensity, enabling both categorical classification and continuous gradient modeling. All 144 samples are ethically sourced (100% human recordings by the author with explicit consent), unprocessed to preserve tonal fidelity, and include comprehensive metadata for reproducible research.

Research Applications: TonalityPrint may function as a precision-tuning resource for investigating: inference-time prosodic alignment in reasoning-based voice models, ambiguity-aware dialogue systems, style-conditioned speech synthesis, embodied AI voice-appearance synchrony, and socio-pragmatic attention mechanisms (e.g. Truthfulness Calibration- Potentially aligning model confidence scores with vocal doubt (Ambivalence) to prevent deceptive confidence,  audibly signally uncertainty when hallucinating). As a controlled single-speaker corpus, it complements rather than replaces large-scale multi-speaker datasets - designed for architectural development, feature extraction research, and transfer learning evaluation, opens new research directions

Limitations: Findings are derived from a single speaker in one professional context and require independent validation across speakers, cultures, and domains. Observational conversion data reflect correlation rather than causation, and are subject to numerous confounding variables. The dataset aims to provide a precision-tuning resource for investigating a replicable annotation methodology and controlled substrate for validation studies but does not itself constitute proof of the Tonality as Attention framework or downstream efficacy.

Availability: Dataset available now for immediate precision-tuning experiments with full methodological documentation released under CC BY-NC 4.0 at Zenodo https://doi.org/10.5281/zenodo.17913895,  with commercial licensing available. The accompanying Tonality as Attention white paper is available separately at https://doi.org/10.5281/zenodo.17410581. 

TonalityPrint aims to address a critical gap in voice AI training data by moving beyond discrete emotion recognition to capture functional tonal intent, including ambivalent prosodic signals as potentially essential nuances for inference-time alignment.