Canadian Parliament Open Data and Embedded Vector Knowledge Base: InsightToast (UIST '26) Supplemental Dataset

📄 Paper: https://doi.org/10.1145/3830398.3830522
💻 Code: https://github.com/ubixgroup/InsightToast

This dataset provides a collection of 43,878 open legislative records from the Canadian House of Commons, spanning 1994 to 2025, covering document types such as debates (Hansard), legislative bills, committee reports, petitions, recorded votes, order papers, and Library of Parliament research publications. It is released at three stages of the same pipeline: the raw collected documents, a processed corpus that has been passed through an ETL pipeline and chunked for use in a retrieval-augmented generation (RAG) pipeline, and an embedded, pre-indexed vector store built from that processed corpus.

The dataset supports the development and user study of the InsightToast project. The embedded form can be placed directly into the project's codebase, open sourced at https://github.com/ubixgroup/InsightToast, where it serves as the knowledge base when InsightToast operates as a proactive side-channel assistant in policy discussion meetings.

Usage in the InsightToast Backend

Only qdrant_storage.tar.gz is required to run the backend. Extract it into the root of the InsightToast project, so that qdrant_storage/ sits alongside docker-compose.yml, and start the services:

tar -xzf qdrant_storage.tar.gz -C /path/to/InsightToast/
cd /path/to/InsightToast
docker compose up --build -d

Contents

Verify the archives after download:

sha256sum -c SHA256SUMS.txt

Source Corpus: Raw Parliamentary Documents

parliament_data.tar.gz contains open legislative documents collected across the range of record types produced by the Canadian House of Commons and the Library of Parliament, including the proceedings of debate, the texts of bills as tabled, the reports of standing committees, the petitions presented to the House, and the recorded outcomes of divisions. The documents were assembled from three public sources: the House of Commons (ourcommons.ca, including the Legislative Information service, LEGISinfo), the Library of Parliament of Canada (parl-gc.primo.exlibrisgroup.com), and the independent open-data mirror openparliament.ca. No private, personal, or access-restricted material is included; Member of Parliament profiles contain only information published in an official public capacity.

• Total: 43,878 parliamentary records, distributed across 57,124 files (43,634 .txt + 13,490 .pdf). Each record is provided as extracted plain text, with the source PDF alongside it under the same basename where one exists, so records that carry both formats account for two files.
• Temporal coverage: documents span the 35th through 45th Parliaments. Debates run from 1994-01-17 to 2025-10-10; coverage of the other record types varies by type.
• Languages: the source records are bilingual (English and French), as published by Parliament.
• Filename conventions: debates use sitting dates (YYYY-MM-DD-<sitting>.txt); bills, journals, order papers, and votes use <parliament>-<session>-<number> (for example, 37-1-C-10); petitions and Library of Parliament publications use slugified titles.

See parliament_data.STRUCTURE.txt for the full breakdown, including all committee subfolders.

Processed Corpus: Cleaned and Chunked Records

parliament_data_processed.tar.gz contains the same corpus after the ETL pipeline that turns each document into retrievable units. Documents are parsed from their metadata header and routed by document_type. Bilingual records are language-identified with langdetect and reduced to their English content by an LLM pass. PDFs are structurally partitioned with unstructured's partition_pdf into text, table, and image elements; table elements are linearized into prose by an LLM, and salient figures are captioned, written to images/, and linked from their chunk via has_images. Records whose content is inherently tabular, namely recorded votes, petitions, and Member profiles, are rendered into natural-language narratives to keep them semantically retrievable rather than reducing to fields. Type-specific pre-segmentation then splits debates on speaker turns, preserving the speaker, section, and timestamp, before every stream passes through a RecursiveCharacterTextSplitter (1,000 characters, 200 overlap); each resulting chunk is tagged with its content_type (text, table, or image) and its provenance.

The output is one JSON document per record, holding the record's metadata and its ordered list of chunks:

{
  "metadata": { "document_type": "debate", "session": "35-1", "date": "1994-01-17",
                "language": "en", "source_file": "debates/1994-01-17-1.txt",
                "source_url": "...", "total_chunks": 51 },
  "chunks": [
    { "chunk_id": 0, "content": "...", "metadata": { "speaker": "...", "section": "...",
                                                     "content_type": "text", "has_images": false } }
  ]
}

• Volume: 42,014 JSON records and 2,170 extracted images, totaling 6,379,732 chunks, which is the set embedded into the knowledge base below.
• Traceability: every record's metadata.source_file links back to its file in the raw corpus.
• Coverage: most record types are processed in full; committee-report and library are present as subsets of the raw corpus. See parliament_data_processed.STRUCTURE.txt for per-category counts.

Indexed Knowledge Base

qdrant_storage.tar.gz contains the processed corpus above embedded and indexed into Qdrant, ready to serve as the retrieval backend with no further processing.

• Embedding model: embeddinggemma:latest (served via Ollama), 768 dimensions, cosine distance.
• Qdrant version: built with qdrant/qdrant:v1.13.6. Qdrant's on-disk format is version-specific, so load this snapshot with the same or a compatible version to avoid migration issues.

Inspecting the knowledge base on its own

To browse or query the collections independently of the InsightToast stack, mount the extracted directory into a standalone Qdrant container:

tar -xzf qdrant_storage.tar.gz            # -> ./qdrant_storage/

docker run -p 6333:6333 \
  -v "$(pwd)/qdrant_storage:/qdrant/storage" \
  qdrant/qdrant:v1.13.6

# reports the collection and its point count:
curl http://localhost:6333/collections/parliament_documents

License

The parliamentary source documents and the embeddings derived from them are released under the Open Government Licence – Canada (https://open.canada.ca/en/open-government-licence-canada). When reusing them, include the attribution:

Contains information licensed under the Open Government Licence – Canada.

The original contributions in this record, namely the curation and compilation of the corpus, the ingestion and indexing pipeline output, and the accompanying documentation, are released under CC-BY 4.0.

Citation

Please cite the accompanying paper if you use this dataset in your work.

@inproceedings{abolnejadian2026insighttoast,
  author    = {Abolnejadian, Mohammad and Brehmer, Matthew},
  title     = {InsightToast: Proactive Information Retrieval and Glanceable Visualization in the Side Channel of Data-Rich Meetings},
  year      = {2026},
  isbn      = {979-8-4007-2856-3/2026/11},
  publisher = {Association for Computing Machinery},
  address   = {New York, NY, USA},
  doi       = {10.1145/3830398.3830522},
  booktitle = {Proceedings of the 39th Annual ACM Symposium on User Interface Software and Technology},
  series    = {UIST '26},
  location  = {Detroit, MI, USA}
}