Published January 7, 2026 | Version v2
Software Open

A Benchmark for Citation-Grounded Geographic QA

Authors/Creators

Description

GroundedGeo

Updated release of GroundedGeo, a benchmark for citation-grounded, freshness-aware, and conflict-aware geographic QA.

Dataset

  1. 200 queries across 5 hard-case buckets (40 each):
    • Boundary-adjacent queries (near county/state lines)
    • Ambiguous name queries (place names in 10+ states)
    • Overlapping jurisdiction queries
    • Freshness-sensitive queries
    • Conflict queries (official vs. community sources)
  2. 147 dev / 53 frozen test split
  3. 100% evidence coverage: every query includes gold_evidence with URLs, source types, retrieval dates, and publication dates
  4. 85 multi-source queries for testing source preference & conflict handling

Frozen Benchmark Results (Test Split)

System Accuracy  Conflict Handled
Closed-Book LLM  17.0% 11.1%
Naïve RAG  79.2%  11.1%
Official-First RAG 94.3%  100.0%
Freshness-Filter RAG 79.2% 11.1%
Conflict-Aware RAG 92.5%  88.9%


Key Finding:

Naïve RAG reaches 79.2% accuracy but fails on conflicting sources (11.1% conflict-handled). Adding official-source ranking improves overall accuracy to 94.3% and raises conflict handling to 100%.

Artifacts:

  • groundedgeo_paper_v2.pdf — Full research paper 
  • data/groundedgeo_v1.0.json — Full dataset (schema + metadata)
  • data/groundedgeo_v1.0.jsonl — HuggingFace format
  • data/groundedgeo_v1.0.csv — Spreadsheet view
  • results/final_results.json — Frozen test metrics
  • results/error_analysis.json — Error analysis

Reproducibility

 Run baselines + evaluation from the repo (eval/) using the included splits and configs.

How to Cite

APA: Pandya, N. (2026).

GroundedGeo: A Benchmark for Citation-Grounded, Freshness-Aware, and Conflict-Aware Geographic QA.

Zenodo. https://doi.org/10.5281/zenodo.18142378

BibTeX:

@misc{pandya2025groundedgeo, title={GroundedGeo: A Benchmark for Citation-Grounded, Freshness-Aware, and Conflict-Aware Geographic QA}, author={Pandya, Nidhi}, year={2026}, doi={10.5281/zenodo.18142378}, url={https://github.com/nidhip1611/GroundedGeo}, publisher={Zenodo} }

Links

Author

Nidhi Pandya

Pace University, Seidenberg School of Computer Science

ORCID: 0009-0001-7313-5892

License

  • Dataset: CC BY 4.0
  • Code: MIT License

 

 

Files

groundedgeo_paper.pdf

Files (898.0 kB)

Name Size Download all
md5:72e65653623c2f700cfd645eb57f5c5f
449.0 kB Preview Download
md5:72e65653623c2f700cfd645eb57f5c5f
449.0 kB Preview Download

Additional details

Related works

Is supplement to
Software: https://github.com/nidhip1611/GroundedGeo/tree/v1.0.0 (URL)

Software

Repository URL
https://github.com/nidhip1611/GroundedGeo