DEFT-RA: Defence-Fitted Targeted Reasoning Assessment for Attribution-Based Plausibility Filtering of LLM-Generated Hypotheses in Defence Research and Development
Description
This position paper introduces DEFT-RA (Defence-Fitted Targeted Reasoning Assessment), a methodological framework designed to address the critical challenge of triaging Large Language Model (LLM) generated hypotheses within the high-stakes environment of defence R&D. Unlike open science, defence research is characterized by multi-year verification cycles and severe asymmetric inductive risks, where the cost of pursuing implausible conjectures is exceptionally high.
DEFT-RA moves beyond standard post-hoc filtering by implementing a structured, document-section-level attribution layer across a specialized ontology encompassing Technology Readiness Level (TRL) statements, capability requirements, threat models, CONOPS, and ICD-203-standardized intelligence products. The architecture integrates Shapley-based attribution with Bayesian evidence fusion and conformal prediction to provide calibrated plausibility sets and explicit audit trails. A central contribution is the typology of nine "implausibility patterns" (such as TRL overreach and paradigm drift) systematically grounded in the philosophy of science. By making the reasoning traces of LLM outputs inspectable and value-laden trade-offs explicit, DEFT-RA provides a rigorous scaffold for the responsible integration of automated discovery into complex defence pipelines.
Files
DEFT_RA_Defence_Hypothesis_Triage.pdf
Files
(334.9 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:28c2a7cf8a9da51c5ac33f3c27d9ca0e
|
334.9 kB | Preview Download |