Searching with AI: data from a comparison of AI-assisted search tools and Boolean database searching (2026)

This dataset documents an experiment in which one research question — how AI-based systems are used in scholarly literature search, and with what implications for research, higher education and university libraries — was searched in two ways: with a Boolean strategy across seven bibliographic databases (Scopus, Web of Science, LISTA, LISA, ERIC, ACM Digital Library, Academic Search Complete), and with four AI search tools that retrieve before answering (Claude Pro, Microsoft Copilot, Keenious, Scopus AI) under a three-tier prompt protocol. Two further conditions used the Consensus index (through an MCP connector and through its own interface). A control ran the same prompts on OpenAI GPT-5.1 without retrieval access.

All 1,584 unique records are included, each checked for whether the cited work exists and (for those that did) graded 0–5 for relevance by a single grader. The package contains the master record table, three readable lists (CSV, RIS, Excel), the identifier and existence checks, the raw exports of the AI conditions (bibliographic fields only), verbatim transcripts and prompts, a description of the models and tools used, the review protocol, and scripts that regenerate every figure in the accompanying master data document. The file README.md provides a fuller introduction to the materials, so please start there. 

Please note, that a large part of the work of building and checking this package was done with generative AI (Anthropic's Claude): the scripts, deduplication and matching, most identifier and existence checks, and the documentation. All relevance grades and all decisions are the author's own. README.md, section "How this package was made", states what was done by AI, what by hand, and what that means for any error a user may find.

The experiment is part of the scoping review (to be published separately later) "Searching with AI: Practices and Implications for Scholarly Discovery and University Libraries" (protocol registration: 
https://doi.org/10.17605/OSF.IO/AKZ7R). The Boolean database searches were run by the information specialists of the scoping review team: Minna Suikka (Scopus), Anne Ojanen (Web of Science), Sanna Raninen (LISA), Päivi Helminen (ERIC), Katri Larmo (ACM Digital Library) and Rosa Venäläinen (Academic Search Complete). Warm thanks to all of them.