Between Randomness and Arbitrariness: Some Lessons for Reliable Machine Learning at Scale (The Short Version)

This document contains the introductory chapter of the dissertation, “Between Randomness and Arbitrariness: Some Lessons for Reliable Machine Learning at Scale,” which was completed in fulfillment of the requirements for a Ph.D. in Computer Science at Cornell University. This dissertation articulates a research vision for a new field of scholarship at the intersection of machine learning, law, and policy. The introduction outlines the three parts of the dissertation, which make contributions in this field with respect to three overarching themes: (1) locating and mitigating sources of arbitrariness in machine learning, (2) taming randomness in scalable, reliable machine learning algorithms, and (3) developing legally cognizable generative-AI evaluations. The research described within these three themes, especially the work on generative-AI evaluations, has had a concrete impact on legal scholarship and U.S. AI policy.