Rubric for review of AI-readiness evaluation criteria

This rubric provides researchers with a structured framework to assess whether a dataset is sufficiently prepared for AI/ML applications across seven core domains, with a focus on data quality and reproducibility. Evaluators score specific criteria on a 0–2 scale to generate overall readiness metrics, collaborating with domain experts for highly specialized assessments. The methodology introduces critical "gating" thresholds in areas such as FAIRness, Provenance, Standards, and Ethics, where failing to meet minimum requirements results in an automatic overall evaluation failure. To guide reviewers, the document outlines precise scoring calculations and includes a detailed glossary defining essential infrastructure, ethical, and technical terms. Ultimately, this tool serves as a rigorous checklist to ensure datasets are reliably and ethically prepared for machine learning applications. The rubric accompanies our “AI-readiness Criteria for Biomedical Data,” available at https://www.biorxiv.org/content/10.1101/2024.10.23.619844.