To what extent does the accuracy of multi-step retrieval pipelines for multi-hop QA degrade under noisy or adv

This paper proposes a framework for quantitatively evaluating interactive LLMs such as ChatGPT using publicly available data sets. We carry out an extensive technical evaluation of ChatGPT using 23 data sets covering 8 different common NLP application tasks. We evaluate the multitask, multilingual and multi-modal aspects of ChatGPT based on these data sets and a newly designed multimodal dataset. We find that ChatGPT outperforms LLMs with zero-shot learning on most tasks and even outperforms fine-tuned models on some tasks. We find that it is better at understanding non-Latin script languages

Research goal: To what extent does the accuracy of multi-step retrieval pipelines for multi-hop QA degrade under noisy or adversarial intermediate contexts, and does this degradation scale with the number of hops (2 vs 5) across GPT-4 and open-source models?

Autonomous synthesis report generated by SOVEREIGN Research Kernel. Tribunal consensus score: 9.2/10.