From Occlusion to Transparency: An Occlusion-Based Explainability Approach for Legal Judgment Prediction in Switzerland

Natural Language Processing (NLP) models have been used for more and more complex tasks such as Legal Judgment Prediction (LJP). A LJP model predicts the outcome of a legal case by utilizing its facts. This increasing deployment of Artificial Intelligence (AI) in high-stakes domains such as law and the involvement of sensitive data has increased the need for understanding such systems. We propose a multilingual occlusion-based ex- plainability approach for LJP in Switzerland and conduct a study on the bias using Lower Court Insertion (LCI). We evaluate our results using different explainability metrics introduced in this thesis and by comparing them to high-quality Legal Expert Annotations using Inter Annotator Agreement. Our findings show that the model has a varying understanding of the semantic meaning and context of the facts section, and struggles to distin- guish between legally relevant and irrelevant sentences. We also found that the insertion of a different lower court can have an effect on the prediction, but observed no distinct effects based on legal areas, cantons, or regions. However, we did identify a language disparity with Italian performing worse than the other languages due to representation inequality in the training data, which could lead to potential biases in the prediction in multilingual regions of Switzerland. Our results highlight the challenges and limitations of using NLP in the judicial field and the importance of addressing concerns about fairness, transparency, and potential bias in the development and use of NLP systems. The use of explainable artificial intelligence (XAI) techniques, such as occlusion and LCI, can help provide insight into the decision-making processes of NLP systems and identify areas for improvement. Finally, we identify areas for future research and development in this field in order to address the remaining limitations and challenges.