Reproducibility data for a study of regulatory statements in EU legislation

Reproducibility data for a quantitative study on EU legislation

The files in this repository were generated or used in a pipeline of analysis operations on EU legislation published between 1971 and 2022. The project is called the Nature of EU Rules which seeks to analyse the "strictness" and density of EU regulations over time and by legal policy area. The data has been made available to help make the results of our study reproducible by other researchers. The underlying data used in the study has also been published in this repository.

File descriptions

1. complete_training_data.csv
   • This file is training data for binary classification of specific sentences in EU legislation as either regulatory in nature (constituting a legal obligation for some agent) or not (called a constitutive statement). The sentences have been labelled by EU law professors from Aarhus University in Denmark and Radboud University in the Netherlands
   • Note: The file also contains columns for identifying the specific agent being regulated (to which the legal obligation applies) in each sentence. However, this information has not been used in the study
2. extracted_sentences_classified_1971_2022.csv
   • List of sentences extracted from EU legislation documents
   • Classification results for individual sentences whether each is regulatory or not. There are two columns recording the classification results, one for a rule-based approach (using grammatical dependency parsing) and one for a LegalBERT classification approach.
3. inlegal_bert_xgboost_classifier.json
   • Trained binary classification model for classifying sentences as regulatory or not (based on InlegalBERT).
   • Note: this model is trained on the file 'complete_training_data.csv' in this Zenodo repo
   • Model was trained using this script and used by these scripts: one, two
4. metadata_enriched.csv
   • Metadata file from this repository but enriched with additional columns one of which is the count of regulatory sentences in each individual document
   • File is generated by this script
   • File is used by this script
5. classification_results_all_algorithms_test_set.csv
   • classification results of each sentence in the test set containing 1451 sentences (20% of training set)
   • according to both the fine-tuned Legal-BERT model and the dependency parsing (rule-based) algorithm
   • also contains the ground truth labels

Github repositories relevant to this analysis

The Python scripts in the following Github repositories were responsible for generating the data files in this Zenodo repository. The first repository listed is the core one for running the pipeline to classify and quantitatively analyse legal obligations in EU legislation. The other listed Github repositories represent components or steps of the pipeline.

• http://github.com/nature-of-eu-rules/eu-legislation-strictness-analysis
  • http://github.com/nature-of-eu-rules/data-extraction
  • http://github.com/nature-of-eu-rules/data-preprocessing
  • http://github.com/nature-of-eu-rules/regulatory-statement-classification