Published May 14, 2026 | Version v0.3.0
Software Open

AI-as-Infrastructure/aiinfra-atlas: Inter-rater upgrade

Authors/Creators

Description

This release prepares ATLAS for the inter-rater reliability focus group study (15 paid reviewers × 20 ratings each, 100 prompts).

Highlights since v0.2.5:

  • Inter-rater session seeding (make seed / seed-dry / seed-reset) with 100 curated 1901 parliamentary prompts across AU/NZ/UK/all corpora.
  • Count-aware allocator (sort by inter_rater_count ASC, sha256 tiebreak)
    • 60 s per-user cache: delivers the gold-standard 1:1 design of exactly 3 independent ratings per prompt under full attendance.
  • INTER_RATER_DEFAULT_UI mode: reviewers land directly on the inter-rater workflow with the New Session button and privacy toggle hidden.
  • Per-scale comment fields on all 6 Likert scales, mandatory for extreme ratings (1, 2, 5).
  • INTER_RATER_SESSIONS_PER_USER bumped 10 -> 20 for the focus-group workload.
  • Security: python-jose -> PyJWT migration, FastAPI security headers, prod-gated debug endpoint, dependency bumps for python-multipart, python-dotenv, langchain-text-splitters, langchain-openai, pytest.
  • Documentation: canonical 'Default Focus-Group Study Configuration' section with a 10-item researcher checklist; seed commands in README.

Files

AI-as-Infrastructure/aiinfra-atlas-v0.3.0.zip

Files (841.7 kB)

Name Size Download all
md5:44aa0ea5e6084328a4a6f01a8793df6b
841.7 kB Preview Download

Additional details

Related works