Published November 5, 2026 | Version v6

OpenScientist: Supplementary Case Study Data

  • 1. Washington University
  • 2. UK Dementia Research Institute, University College London
  • 3. Lawrence Berkeley National Laboratory

Description

Supplementary data for "OpenScientist: evaluating an open agentic AI co-scientist to accelerate biomedical discovery." This dataset contains research logs, knowledge states, provenance files, and generated figures for seven case studies spanning neurodegenerative disease transcriptomics, multiple myeloma genomics, survival proteomics, drug side effect analysis, and more. Each case study includes the full agent iteration log, configuration, and final report.

Input datasets are included for three case studies where data sharing is permitted: case study 3 (neurofibrillary tangle transcriptomics), case study 4 (multiple myeloma RNA-seq), and case study 7 (drug side effect frequencies from clinical trials). 

Version 2 (August 2026) adds supplementary material for case study 3 (neurofibrillary tangle transcriptomics): 60 additional independent runs of the same question and dataset, comparing three agent/model configurations — Claude Code with Claude Opus 4.8, and the omp harness with Kimi K3 and GLM 5.2 — each run 10 times online and 10 times in a fully air-gapped configuration with no network access and literature served from a local 40-million-article MEDLINE mirror.

Files: ..._supplemental_online_30runs_data.tar.gz and ..._supplemental_airgapped_30runs_data.tar.gz each contain 30 runs with reports (Markdown, PDF, HTML), all generated figures, per-iteration transcripts, the exact prompt, and full version/commit provenance. ..._supplemental_60runs_metrics.csv gives per-run runtime, tool-call counts, literature searches, findings recorded and token usage for all 60 runs.

Version 3 (September 2026) adds two further supplementary sets for case study 3 (neurofibrillary tangle transcriptomics).

..._supplemental_reproducibility_10runs_data.tar.gz contains 10 independent runs of the same question on Claude Opus 4.6, all run on 2026-03-08 with byte-identical inputs (the research question and the input .h5ad were SHA-256 verified identical across all 10) and a 10-iteration budget. Each run includes the report (Markdown, PDF, HTML), knowledge_state.json with structured findings and literature list, per-iteration transcripts, and the job UUID. These runs quantify run-to-run variability under fixed inputs.

..._supplemental_genomic_skill_comparison_data.tar.gz contains 30 runs — 15 using the original genomics Agent Skill and 15 using a revised version — with the two skill definitions included alongside the jobs. The revised skill adds explicit guidance on donor-level pseudobulk aggregation and mixed-effect modelling; the paired arms allow the effect of that guidance on the analysis the agent chooses to be assessed directly. Each run includes reports (Markdown, PDF, HTML), per-iteration provenance transcripts, and container logs.

Version 4 (October 2026) adds the run sets generated for the second revision of the manuscript, renumbers the case-study files to match the revised manuscript (the suvorexant proteomics analysis is now Case Study 5; neurofibromatosis is Case Study 6, drug side effects Case Study 7, and the CSF Aβ42/40 prediction model Case Study 8), and adds RUNS_MANIFEST.csv, which maps every job in this record to its experiment, supplementary table and row label. Run IDs quoted in the supplementary tables are the first 8 hexadecimal characters of the job ID.

New archives, all for Case Study 3 (neurofibrillary tangle transcriptomics; same research question and input file as the existing archives, OpenScientist commit fac6971, Claude Opus 4.6, run 2026-10-18 with a 10-iteration budget): ..._supplemental_iteration_controls_10runs_data.tar.gz (10 replicate runs on unmodified code; Experiment 4 and the control arm of the two ablations), ..._supplemental_knowledge_state_ablation_10runs_data.tar.gz (10 runs in which the knowledge-state summary and outline were withheld from the agent; Experiment 5; the patched orchestrator file is included as ..._knowledge_state_ablation_iteration.py.patched), and ..._supplemental_formula_removed_10runs_data.tar.gz (10 runs with the hypothesis-prioritization scoring rules removed from the agent skills and system prompt; OpenScientist pull request 310). Each run directory contains the report (Markdown, PDF, HTML), knowledge_state.json, claude_iterations.log and per-iteration provenance transcripts; the input .h5ad is not duplicated.

Development runs behind the per-case-study run tables: Case_study_1_SEABIRD_supplemental_development_11runs_data.tar.gz (the 11 SEABIRD runs of Supplementary Table 2; input data not included), Case_study_2_Survival_Proteomics_supplemental_development_3runs_data.tar.gz (3 runs; input data not included) and Case_study_3_NFT_Transcriptomics_supplemental_exploratory_2runs_data.tar.gz (the two exploratory runs that preceded the benchmark run a66ae54c).

Version added 5 November 2026: analysis_code.zip, the analysis code behind the manuscript figures and tables (Case Study 3 scoring and Extended Data Figs. 1 and 2 scripts and the self-contained Figure 2 CONSORT renderer), with a README.

Files

analysis_code.zip

Files (1.1 GB)

Name Size
md5:f28b73767b82f6a49d2d2a833ebbc9ff
44.6 kB Preview Download
md5:85bdc86c2915967595cf9da476d6ea61
7.2 MB Download
md5:86cdd089dc4cd3ac22b01b49dcba6c64
59.8 kB Preview Download
md5:39823a2fd5632e29ab8b98ab5cdac18e
98.0 MB Download
md5:1a1f6615aceea3a87f1f141adeae12d3
6.5 MB Download
md5:0362075e58835c2653f44b4e0c5b6901
54.6 kB Preview Download
md5:160a8633803dbf028e169cb0fd20444e
20.6 MB Download
md5:943aaa77566aa4b80271b9234b357ca6
250.0 MB Download
md5:3fa00ae2f317e515958f266881f811e4
65.7 kB Preview Download
md5:842b7af06b9bf8fc7e393a97dd559a88
14.1 kB Preview Download
md5:1d9aa236dd5be7e6b6e1cc67885af8a5
111.2 MB Download
md5:e9e2a67c9a99dc1ac2edfcc91d47103f
26.7 MB Download
md5:be1b629a7d964702f415b3a7a50c387d
86.2 MB Download
md5:8a52e90b8419261fd3c55a3dca01a2c2
61.3 MB Download
md5:0d959422e29c50b3eabd996f7d889510
76.4 MB Download
md5:7d29385a1bdca796f096b7295796d04d
86.9 MB Download
md5:40355e31302d86d67dfd7e17e0a66018
22.7 kB Download
md5:c6c3ce9850553f9ef257ef851beee56a
151.5 MB Download
md5:76a2fdad67542501f3f707d3f4b79f93
2.7 MB Download
md5:154ce6ac1e20eccf5023d83b303eca80
75.3 MB Download
md5:c66b1f86b0671d176f56355f1c0bba12
44.0 kB Preview Download
md5:4b36620d16a42565c358d70e681a7304
61.0 MB Download
md5:edae450809c0399e07b64ee5efc1ca95
135.5 kB Preview Download
md5:6ecac394445fb2d2c1e76894aaf8e0a7
3.9 MB Download
md5:477c331b37c01a66b2bee632c0bb488d
77.6 kB Preview Download
md5:e1908e13467189f87d40d7535f10a6f4
8.9 MB Download
md5:0258f4043a58c27edc7d0e7e0849936f
87.4 kB Preview Download
md5:36b4d245b928e437888f63ee57a26167
6.2 MB Download
md5:4b481476a34f9c04ec1e4acad0793e9f
67.4 kB Preview Download
md5:81ad22cfae6ce6e2bba2cd1183f4a63b
33.1 kB Preview Download