Mediknow - A Malayalam Cancer Question Answering System

Abstract

This paper introduces "MediKnow," a pioneering Malayalam Question Answering System designed to address the scarcity of generative answer works in the realm of healthcare information accessibility, specifically tailored for cancer-related queries. The dearth of such systems in Dravidian languages, particularly Malayalam, has motivated the development of a robust solution. Leveraging advanced Natural Language Processing (NLP) techniques, including OpenAI models and FAISS for efficient vector storage, MediKnow employs a specialized Malayalam language model to navigate the intricacies of the Dravidian linguistic context. The processing pipeline encompasses document loading, text splitting, and embeddings, enhancing the system's capacity to comprehend and accurately respond to a diverse range of cancer-related questions. This work underscores the critical need for bridging the gap in generative answer works for Dravidian languages, highlighting the specific challenges posed by the Malayalam language due to its complexity. Beyond providing accessible information, MediKnow exemplifies the efficacy of employing state-of-the-art NLP technologies to address linguistic nuances. The paper evaluates the system's performance on a dataset of cancer-related questions, demonstrating its ability to deliver accurate and informative answers. The innovative approach presented herein contributes to the advancement of NLP capabilities in non-English languages, particularly focusing on healthcare-related information retrieval. The development and deployment of "MediKnow" signify a significant stride in tackling linguistic and domain-specific challenges in cancer-related question answering, ultimately making critical healthcare information more accessible to Malayalam speakers.