There is a newer version of the record available.

Published January 13, 2026 | Version v0.2.2
Software Open

AI-as-Infrastructure/aiinfra-atlas: Feedback inter-rating

Authors/Creators

Description

This release adds functionality for inter-rating of user feedback. Refer to /docs/inter_rater.md for configuration details.

Major Features:

  • Numbered inter-rater annotations ([inter-rating-1], [inter-rating-2], [inter-rating-3])
  • Prevents annotation overwrites with unique IDs per rater
  • Async event loop compatibility for concurrent users
  • Enhanced documentation with allocation algorithm details

Key Changes:

  • Made submit_span_annotation() async for proper FastAPI integration
  • Include inter_rater_number in annotation IDs and metadata
  • Use async httpx.AsyncClient for non-blocking Phoenix queries
  • Deterministic user-specific session allocation

Fixes:

  • Fixed inter_rater_number always being null (GitHub issue #43)
  • Fixed annotation overwrites when multiple raters rate same session
  • Fixed async/await event loop conflicts in FastAPI

Documentation:

  • Inter-rating configuration guidelines for different team sizes
  • Coverage calculation formulas and examples
  • Detailed allocation algorithm explanation
  • Workload estimation guidance

Related: GitHub issue #43

Files

AI-as-Infrastructure/aiinfra-atlas-v0.2.2.zip

Files (692.9 kB)

Name Size Download all
md5:4e8af07fa23baf46fb118a1eed557f93
692.9 kB Preview Download

Additional details

Related works