residential_London_comments
Description
'comments.csv' is a dataset of 30,393 public comments left in response to planning applications for new residential developments in London.
The dataset includes comments made in response to planning applications filed between 1/1/2021 and 1/1/2025. It covers eight of Londons local authority districts: City of London, Westminster, Lambeth, Southwark, Newham, Barnet, Brent and Ealing.
The dataset contains the following columns:
| Name | dtype | Description |
| id | int | id generated during scraping |
| council | string | The local authority district (LAD) of the planning application |
| comment_id | string | The comment ID used by the LAD |
| application_id | string | The planning application ID used by the LAD (sometimes referred to as planning reference) |
| address | string | Comment address |
| stance | string | Registered stance to planning applciation ('objects', 'support', 'neutral') |
| data | datetime | The date and time the comment was left online |
| comment_text | string | Freetext of the comment |
| add_date | datetime | The date and time the comment was webscraped |
| lat | float | Latitude extracted from the comment address using Nominatum |
| lon | float | Longitude extracted from the comment address using Nominatum |
| cleaned_comment_text | string | Freetext of the comment with proper nouns (names and addresses) removed |
| lsoa_code | string | The LSOA code of the comment |
| sentiment_score | float | The sentiment score of the free text |
The dataset is complete, apart from the addresses ('address', 'lat', 'lon', 'lsoa_code') which are missing for 20% of the entries. This is due to the council websites not recording the addresses of all comments.
The column 'application_id' linked this dataset to the Planning London Datahub - the GLA dataset of all planning applications filed in London.
This dataset is used in our analysis of NIMBY responses to planning applications - you can find out more through the GitHub repo: Bea-Taylor/nimby_nlp.
The additional datasets of 'comments_with_topics.csv' and 'topic_names.csv' record the topics generated using BERTopic.
Files
Restricted
The record is publicly accessible, but files are restricted. Log in to check if you have access.
Additional details
Related works
- Is cited by
- Conference paper: 10.5281/zenodo.19420989 (DOI)
Funding
- UK Research and Innovation
- AI for Collective Intelligence (AI4CI) EP/Y028392/1
Dates
- Updated
-
2026-06-02
Software
- Repository URL
- https://github.com/Bea-Taylor/nimby_nlp
- Programming language
- Python
- Development Status
- Active