Pramana: Fine-Tuning Large Language Models for Epistemic Reasoning through Navya-Nyaya

Overview

Pramana introduces the first large language models fine-tuned on explicit Navya-Nyaya epistemological methodology—a 2,500-year-old Indian logical reasoning framework. This work bridges ancient epistemology with modern AI to address the fundamental epistemic gap in LLMs: the inability to ground claims in traceable evidence sources, distinguish valid knowledge from pattern-matching, and express appropriate epistemic humility.

Core Innovation

Unlike generic chain-of-thought prompting which relies on implicit reasoning patterns, Pramana enforces structured 6-phase methodology:

1. Samshaya (Doubt Analysis): Classifies uncertainty into 5 taxonomic categories
2. Pramana (Evidence Sources): Mandates explicit grounding in 4 valid knowledge sources (Pratyaksha/perception, Anumana/inference, Upamana/comparison, Shabda/testimony)
3. Pancha Avayava (5-Member Syllogism): Constructs formal arguments with universal rules (Vyapti) grounded in concrete examples (Drishtanta)
4. Tarka (Counterfactual Testing): Verifies conclusions via reductio ad absurdum
5. Hetvabhasa (Fallacy Detection): Systematically checks 5 reasoning error types
6. Nirnaya (Ascertainment): Distinguishes definitive knowledge from hypotheses requiring verification

This integration of logic and epistemology provides cognitive scaffolding absent from standard reasoning approaches, preventing conflation of evidence types, forcing explicit universal rule statements, enabling systematic error detection, and maintaining epistemic humility.

Architecture & Training

Models Developed:

• Stage 0 (Proof-of-Concept): Llama-3.2-3B-Instruct fine-tuned on 20 examples
• Stage 1 (Minimum Viable Reasoner): DeepSeek-R1-Distill-Llama-8B fine-tuned on 55 examples

Training Methodology:

• QLoRA (4-bit quantization) for efficient training
• LoRA rank 64, targeting all attention + FFN layers
• Supervised fine-tuning with structured Markdown format
• Training costs: <$1.00 per stage, <0.32 GPU-hours (A100 40GB)
• Datasets span constraint satisfaction, Boolean SAT, multi-step deduction, transitive reasoning, and set operations

Prompt Engineering:

• Explicit format instructions with skeletal template injection
• System prompt establishing Nyaya reasoning engine role
• Critical constraint enforcement via generation parameters

Key Results

Stage 1 Performance:

• 100% semantic correctness (10/10 examples) with 95% CI [0.510, 1.0]
• 40% format adherence (4/10 examples) with 95% CI [0.168, 0.687]
• Zero structure abandonment: Models consistently attempt all 6 phases
• Training loss: 0.350 (Stage 1) vs 0.691 (Stage 0), indicating improved model fit

Critical Finding: Dissociation between semantic correctness (100%) and format adherence (40%) reveals models internalize reasoning content even when strict schema compliance fails. This suggests Nyaya methodology teaches genuine reasoning, not just template-filling.

Ablation Studies:

• Format prompting and temperature interact differently across stages
• Stage 0 optimal: format prompting + temp 0.0 (30% semantic rate)
• Stage 1 optimal: format prompting + temp 0.7 (30% semantic rate)
• Base models show 0% format adherence, confirming Nyaya structure is learned through fine-tuning

Failure Mode Analysis:

• Missing Hetvabhasa section (2 cases): fallacy detection perceived as optional
• Invalid doubt types (2 cases): partial schema learning
• Zero structural errors: strong syntactic learning, semantic constraints need reinforcement

Evaluation Framework

Three-Tier Validation:

1. Tier 1 (Structural): Automated format compliance checking (NyayaStructureValidator)
2. Tier 2 (Content Quality): LLM-as-judge with explicit Nyaya rubric (planned for Stage 2)
3. Tier 3 (Ground Truth): Semantic similarity via sentence-transformers embeddings
4. Tier 4 (Formal Verification): Z3 SMT solver integration (infrastructure exists, not yet applied)

Theoretical Contributions

Bridging Ancient Epistemology with Modern AI:

• First demonstration that Navya-Nyaya structures can be learned by neural networks through fine-tuning
• Unlike Western formal logic (divorced from epistemology), Nyaya integrates logic with explicit knowledge sources
• Addresses "epistemic gap" in LLMs: inability to distinguish valid knowledge from probabilistic associations

Interpretability Advantages:

• Every reasoning step traceable to evidence sources (Pramana)
• Universal rules (Vyapti) grounded in concrete examples (Drishtanta)
• Built-in self-verification (Tarka) and error detection (Hetvabhasa)
• Explicit epistemic status (Nirnaya): knowledge vs. hypothesis

Computational Epistemology:

• Token budget: ~1,250 tokens per solution (3-6× CoT overhead, justified by interpretability)
• Phase dependencies: weak Pramana → invalid reasoning → wrong conclusions
• Quality thresholds: minimum 2 complete syllogisms with universal rules required

Open Science Release

All artifacts publicly available on Hugging Face:

• Models: qbz506/nyaya-llama-3b-stage0, qbz506/nyaya-deepseek-8b-stage1
• Dataset: qbz506/pramana-nyaya-stage1 (55 Nyaya-structured logical problems)
• Demo: qbz506/pramana-nyaya-demo (interactive HuggingFace Space)
• Training infrastructure: Complete codebase with callbacks, validators, evaluators

Limitations & Future Work

Current Limitations:

• Format adherence (40%) below target (≥90%), requires constrained decoding or format-specific rewards
• Limited to formal logic problems, domain expansion needed
• Small evaluation sets (Stage 0: 2 examples, Stage 1: 10 examples)
• Max new tokens truncation (256) affects format parsing

Planned Extensions (Stages 2-4):

• Stage 2: Synthetic scaling to 500 examples with LLM-as-judge quality control
• Stage 3: Group Relative Policy Optimization (GRPO) with composite rewards
• Stage 4: Production deployment with constrained decoding (GBNF), rejection sampling, Z3 verification
• Future: Benchmark on LogicBench, ProntoQA, RuleTaker; frontier model comparison (o1, Claude extended thinking)

Impact & Vision

This work demonstrates that systematic reasoning frameworks can be taught to LLMs through fine-tuning, not just prompt engineering. The long-term vision is developing interpretable, trustworthy AI reasoning systems where every conclusion comes with an auditable trail of justification. As AI systems deploy in high-stakes domains (medical diagnosis, legal reasoning, safety-critical systems), Nyaya-structured reasoning provides explicit phases that can be validated, debugged, and improved—capabilities essential for trustworthy AI.

Invitation for Community Research: This foundation opens pathways for integrating other epistemological frameworks (Mimamsa, Buddhist logic, Western formal logic) into neural architectures, advancing toward AI systems that reason systematically and transparently.

Technical Details

• Paper: 52 pages + appendices, comprehensive treatment of Navya-Nyaya computational formalization
• Related Work: Extensive review of computational Indian logic (Matilal 1985, Burton 2020, Ganeri 2001), LLM reasoning (Wei et al. 2022, Lightman et al. 2023, DeepSeek-AI 2025), hallucination mitigation
• Implementation: Python, Unsloth fine-tuning framework, vLLM deployment, Weights & Biases observability
• Evaluation: Manual + automated validation, semantic similarity metrics, comprehensive failure mode analysis

Citation

Sathish, S. (2026). Pramana: Fine-Tuning Large Language Models for Epistemic Reasoning through Navya-Nyaya. Preprint, University of York.

Keywords: Navya-Nyaya, epistemology, LLM reasoning, interpretability, structured reasoning, Indian logic, hallucination mitigation, computational philosophy