Published January 13, 2026
| Version v0.2.2
Software
Open
AI-as-Infrastructure/aiinfra-atlas: Feedback inter-rating
Authors/Creators
Description
This release adds functionality for inter-rating of user feedback. Refer to /docs/inter_rater.md for configuration details.
Major Features:
- Numbered inter-rater annotations ([inter-rating-1], [inter-rating-2], [inter-rating-3])
- Prevents annotation overwrites with unique IDs per rater
- Async event loop compatibility for concurrent users
- Enhanced documentation with allocation algorithm details
Key Changes:
- Made submit_span_annotation() async for proper FastAPI integration
- Include inter_rater_number in annotation IDs and metadata
- Use async httpx.AsyncClient for non-blocking Phoenix queries
- Deterministic user-specific session allocation
Fixes:
- Fixed inter_rater_number always being null (GitHub issue #43)
- Fixed annotation overwrites when multiple raters rate same session
- Fixed async/await event loop conflicts in FastAPI
Documentation:
- Inter-rating configuration guidelines for different team sizes
- Coverage calculation formulas and examples
- Detailed allocation algorithm explanation
- Workload estimation guidance
Related: GitHub issue #43
Files
AI-as-Infrastructure/aiinfra-atlas-v0.2.2.zip
Files
(692.9 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:4e8af07fa23baf46fb118a1eed557f93
|
692.9 kB | Preview Download |
Additional details
Related works
- Is supplement to
- Software: https://github.com/AI-as-Infrastructure/aiinfra-atlas/tree/v0.2.2 (URL)
Software
- Repository URL
- https://github.com/AI-as-Infrastructure/aiinfra-atlas