Training dataset: reconstructing biological metadata from heterogeneous sources (RNA-seq for ENA submission)

This dataset is designed for training purposes to illustrate the challenges of reconstructing biological metadata from heterogeneous and incomplete sources, in a context close to real-life research workflows.

It supports hands-on workshops focused on metadata curation, data integration, and submission to public repositories such as the European Nucleotide Archive (ENA), using the ERC000011 checklist.

🧬 Context

In many biological projects, metadata are not centrally structured but instead scattered across multiple sources such as emails, spreadsheets, sequencing outputs, and file naming conventions. This dataset reproduces such a situation intentionally.

Participants are expected to:

• identify relevant metadata across multiple files
• resolve inconsistencies and ambiguities
• reconstruct a coherent metadata schema
• prepare a submission compatible with ENA standards

📁 Dataset content

The dataset includes:

• A simulated email from a biologist describing the experiment (incomplete and partially uncertain)
• Illumina sequencing metadata files:
  • RunInfo.xml
  • RunParameters.xml
  • SampleSheet.csv
• A spreadsheet containing partial sample annotations
• Minimal FASTQ files (paired-end) for four samples
• File naming conventions reflecting sequencing output
• Minimal FASTQC files (quality)
• Minimal count matrix for differential analysis

🧪 Experimental design (hidden / to be reconstructed)

The dataset corresponds to an RNA-seq experiment involving:

• 4 samples (2 control, 2 treated)
• Mouse liver tissue
• Oxidative stress treatment

However, this information is intentionally distributed and not explicitly structured in any single file.

⚠️ Known limitations (intentional)

• Missing or ambiguous metadata (e.g. organism confirmation, protocol versions)
• Inconsistencies between sources (e.g. sequencing instrument)
• Lack of controlled vocabulary
• Partial or informal annotations

These limitations are deliberate and serve as discussion points during training.

🎯 Learning objectives

This dataset enables training on:

• metadata extraction and normalization
• identification of missing information
• use of controlled vocabularies and standards
• mapping to ENA submission objects (Study, Sample, Experiment, Run)
• reproducibility and data stewardship best practices

🤖 Metadata capture with madbot

This dataset is also intended to support hands-on training with tools such as madbot.

In many real-world projects, metadata collection is fragmented and often deferred to a single person responsible for data submission. This dataset illustrates the limitations of such workflows.

Using madbot, participants can explore an alternative approach where metadata are:

• captured progressively by different contributors (biologists, engineers, data managers)
• structured from the start using appropriate schemas
• centralized in a single system rather than scattered across files

This highlights how distributing metadata entry across stakeholders improves data quality, reduces ambiguity, and simplifies downstream processes such as submission to public repositories like the European Nucleotide Archive.

🧰 Recommended usage

This dataset is particularly suited for:

• workshops on research data management (RDM)
• training sessions on bioinformatics data submission
• demonstrations of tools such as madbot for metadata structuration and automation

It can be used in group exercises where participants collaboratively reconstruct metadata and prepare submission files.

👨‍🏫 Instructor notes

This dataset is designed to trigger discussion around:

• implicit vs explicit metadata
• technical vs biological metadata separation
• real-world data inconsistencies