AimuniRoslan/OTC-MedGuide: v1.0.0 - Initial Release of OTC MedGuide

Overview

This initial release provides the core dataset and evaluation resources developed for the OTC MedGuide chatbot project. The repository is intended to support reproducible testing and technical evaluation of chatbot systems designed to generate over-the-counter (OTC) medication–related information within a controlled and non-diagnostic scope.

Contents of the Release

OTC Medication Dataset (otc_dataset.json)
A structured JSON dataset containing curated OTC medication information, including generic and brand names, therapeutic categories, common uses, dosage forms, routes of administration, warnings, contraindications, age and pregnancy flags, and safety-related disclaimers. This dataset serves as the primary knowledge source for retrieval-augmented generation (RAG) within the chatbot system.

Evaluation Question Set (evaluation_data.py)
A set of 50 manually curated evaluation questions with corresponding ground truth labels, organized across multiple OTC-related categories. This file defines the reference answers used for automated technical evaluation.

Chatbot Response Outputs (chatbot_answers.json)
A JSON file containing the chatbot-generated responses to the evaluation questions. These responses are used as input for performance analysis and comparison against the ground truth labels.

Evaluation Script (run_evaluation.py)
A Python script that implements the automated technical evaluation pipeline. It computes performance metrics such as accuracy, precision, recall, F1-score, confusion matrix, and ROC-AUC based on comparisons between chatbot responses and ground truth data.

Purpose

The purpose of this release is to provide a transparent and reproducible foundation for evaluating OTC-focused chatbot behavior, with particular attention to response relevance, factual grounding, and safety-related constraints. It is intended primarily for academic and experimental use rather than clinical deployment.

Future Extensions

Potential future improvements include expanding OTC coverage across additional symptom categories, incorporating localization considerations (e.g., region-specific OTC products), and extending the evaluation framework with enhanced reporting, visualization, and reproducibility support.