Hansard Speeches and Sentiment V2.2

Full details are available at https://evanodell.com/projects/datasets/hansard-data

Summary

A public dataset of speeches in the Hansard, stored as a tibble class in RDS files, for the R programming language, and also available as a CSV file. The dataset provides information on every speech made in the House of Commons between the parliament returned from the 1979 general election and the dissolution of parliament for the 2017 general election, with information on the speaking MP, their party, gender, birthdate, starting and finishing dates as an MP, and age at the time of the speech. The dataset also includes all speeches made from 1936 to the dissolution of parliament for the 1979 general election. The post-1979 election dataset is labelled hansard_senti_post_V22 and the pre-1979 election dataset is labelled hansard_senti_pre_V22.

The hansard_senti_post_V22 dataset contains 2,230,357 speeches and 398,815,027 words. The hansard_senti_pre_V22 dataset contains 2,977,461 speeches and 406,103,015 words. It is distributed under a Creative Commons 4.0 BY-SA licence.

Changes in V2.2

* Improvements to file encoding, as mojibake were showing up on some platforms.

* Improvements to spacing to ensure punctuation was followed by a space.

* Dropping the sentiwords library from lexical polarity calculations, as there was very little overlap between the language used in parliament and the Sentiwords dataset, and it takes a very long time to process.

* Added the speaker_office variable, which lists the government or opposition position, if any, held by a speaker.

* Change the name of the hu lexicon to huliu.

* Added UK spellings to the afinn, jockers, nrc and huliu lexicons, to improve compatibility and consistency with the house style used by the Hansard.

Notes

The code and matching data used to generate this dataset is available on Github.

The data used to create this dataset was taken from the parlparse project operated by They Work For You and supported by mySociety.

The dataset is licensed under a Creative Commons Attribution 4.0 International License.

The code used to create this dataset is licensed under an MIT license.

Please contact me if you find any errors in the dataset. The integrity of the public Hansard record is questionable at times, and while I have improved it, the data is presented 'as is'.