Quality Challenges Do Not Escalate Source Fabrication in Large Language Models: A Pre-Registered Cross-Platform Factorial Study

Large language models (LLMs) frequently fabricate non-existent URLs and citations. A prominent hypothesis holds that rejection framing — prompts expressing dissatisfaction and demanding better sources — amplifies this behaviour by increasing the pressure on the model to produce specific references (schema activation lock hypothesis).

We conducted a pre-registered 2×5×7 mixed factorial study across two LLM platforms (ChatGPT gpt-4o; Perplexity sonar-pro) and five rejection-framing arms (BASE, SJR, UPR, NSAF, ECC), with seven conversation depths (k=0–6). Twenty fictional entities per platform (1,400 observations total) were used as probes. The primary outcome was the Fabricated Source Rate (FSR): fabricated URLs divided by total URLs cited. A two-tier validation protocol distinguished URL non-existence (HTTP Tier 1) from content misattribution (TF-IDF Tier 2, 30% stratified sample).

The primary hypothesis (H1: SJR rejection framing amplifies FSR relative to BASE, OR≥1.5) was not supported on either platform (ChatGPT: OR=1.00, p=0.998; Perplexity: OR=1.19, p=0.822). Uncertainty-preserving framing (UPR) produced complete suppression of fabrication at k≥1 on both platforms (0/240 positive events), as did gap-anchoring framing (NSAF) on Perplexity (0/120 events). Platform architecture was the dominant determinant of FSR: ChatGPT averaged FSR=0.075, Perplexity FSR=0.008 (9.2× difference). Content-level validation revealed that Perplexity’s near-zero HTTP fabrication rate masks near-universal content misattribution (95.5% of HTTP-validated URLs pointed to topically related but entity-unspecific pages). The expansion framing arm (ECC) showed a significant platform-differential interaction: amplifier on ChatGPT, suppressor on Perplexity (OR=0.157, p=0.042).

Quality challenge framing does not amplify URL fabrication; rather, the unrestricted baseline request produces maximal fabrication pressure. Uncertainty-preserving and gap-acknowledging framings provide robust, architecture-independent protection. Parametric and retrieval-grounded systems exhibit distinct failure modes that require different mitigation strategies.

Pre-registration: OSF https://osf.io/3es5f/ (v0.4)
Code & Data: https://github.com/glasseymour/rice-p1