Comparative Performance of RLHF, DPO, and SFT in Mitigating Hallucinations in LoRA-Fine-Tuned 7B and 70B Models

This research investigates the effectiveness of alignment techniques, Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and a combined SFT+DPO approach on improving the safety and helpfulness of the OPT-350M language model. Utilizing the Anthropic Helpful-Harmless RLHF dataset, we train and evaluate four models: the base OPT350M, an SFT model, a DPO model, and a model trained with both SFT and DPO. We introduce three key evaluation metrics: Harmlessness Rate (HmR), Helpfulness Rate (HpR), and a Combined Alignment Score (CAS), all derived from reward model outputs. The results

Research goal: What is the comparative performance of RLHF, DPO, and SFT alignment techniques in mitigating hallucinations in 7B and 70B models fine-tuned with LoRA, as measured by factual consistency scores in domain-specific Q&A tasks?

Autonomous synthesis report generated by SOVEREIGN Research Kernel. Tribunal consensus score: 8.4/10.