Examining Algorithmic Fairness for First-Term College Grade Prediction Models Relying on Pre-matriculation Data

Many colleges use AI-powered early warning systems (EWS) to provide support to students as soon as
they start their first semester. However, concerns arise regarding the fairness of an EWS algorithm when
deployed so early in a student’s college journey, especially at institutions with limited data collection
capacity. To empirically address this fairness concern within this context, we developed a GPA prediction
algorithm for the first semester at an urban Japanese private university, relying exclusively on demographic
and pre-college academic data commonly collected by many colleges at matriculation. Then we
assessed the fairness of this prediction model between at-risk and lower-risk student groups. We also examined
whether the use of 33 novel non-academic skill data points, collected within the first three weeks
of matriculation, improves the model. Our analysis found that the model is less predictive for the at-risk
group than their majority counterpart, and the addition of non-academic skill data slightly improved the
model’s predictive performance but did not make the model fairer. Our research underscores that an early
adoption of EWS relying on pre-matriculation data alone may disadvantage at-risk students by potentially
overlooking those who genuinely require assistance.