Machine Unlearning in Large Language Models: The Increasing Need of Systems that Forget

Abstract

The rapid advancement of large language models (LLMs) has revolutionized natural language processing and understanding, but their reliance on vast amounts of diverse and often uncurated data has raised significant concerns about privacy, copyright, model robustness, and alignment with human values. Machine unlearning, the process of systematically removing specific data from a model's training, has emerged as a potential solution to these challenges. However, the majority of research on machine unlearning in LLMs has focused on fine-tuned models, leaving the critical issue of unlearning in pre-trained LLMs largely unexplored.

This thesis investigates the increasing need for machine unlearning in pre-trained LLMs, aiming to identify the most effective techniques, analyze their advantages and limitations, and propose practical frameworks for operationalizing these solutions in real-world applications. Through a comprehensive literature review and analysis, the study reveals that a combination of global and local weight modification methods offers the best balance between unlearning effectiveness, model utility preservation, and computational efficiency. However, these techniques also face challenges such as catastrophic forgetting, incomplete unlearning, and limited interpretability.

To address these challenges, the thesis proposes the development of hybrid unlearning methods, standardized evaluation frameworks, and holistic governance strategies that encompass technical solutions, evaluation methodologies, and collaborative ecosystems. By combining these elements into a unified framework, the study aims to facilitate the effective and responsible deployment of machine unlearning in LLMs, enabling them to adapt to evolving data landscapes and societal expectations.

The findings of this research contribute to the growing body of knowledge on machine unlearning, providing a theoretical foundation and practical recommendations for the development of more ethical, transparent, and accountable AI systems. As LLMs continue to advance and permeate various domains, the insights and frameworks presented in this thesis will play a crucial role in shaping the future of responsible AI innovation.