Records held by the National Archives of Australia with the access status of 'closed' – annual harvests from 2016 to 2025

This dataset contains annual harvests of all records held by the National Archives of Australia that have the access status of 'closed' (withheld from public access). The harvests were run on or about 1 January each year from 2016 to 2025. The aim in saving this data is to enable long-term analysis of the NAA's access examination process.

Australian government records become available for public access after 20 years. But before being opened to the public, records go through a process known as access examination to determine whether they should be withheld, either partially or completely. The grounds for exemption are laid out in the Archives Act and include things like national security and personal privacy. If a record is completely withheld from access, the NAA's database, RecordSearch, records its access status as 'closed'.

This dataset was created by scraping the results of a search for records with the access status of 'closed' from RecordSearch. Additional information, including the reasons each file was closed, and the date of the access decision, was scraped from each item record and added to the data. Some normalisation was applied to the reasons, as the format can vary. The method used is documented in this notebook.

It's important to note that records can be re-examined and their access status can change. Also some 'closed' files are actually 'withheld pending advice' – in these cases a final access decision hasn't been made as the NAA has referred the files to their controlling agencies for advice. This means that this dataset should be treated as providing annual snapshots of an active system, not a cumulative record of closed files. Some of the complexities of the access examination system revealed by this data are discussed in the Inside Story article 'Withheld pending advice'.

The fields available in each CSV file vary a little across the years, but the following fields are always available:

• identifier – the RecordSearch item identifier
• series – series number
• control_symbol – item control symbol
• title – item title
• series_title – series title
• contents_date_str – string describing the date range of the file contents
• contents_start_date – ISO formatted start date of the file contents
• contents_end_date – ISO formatted end date of the file contents
• access_status – current access status ('Closed')
• access_decision_date – ISO formatted date when access decision was made
• reasons – pipe-separated list of normalised reasons for exclusion, these are either references to sections of the Archives Act, such as "33(1)(a)", or a range of other values including "Withheld pending adv", "Closed period",  and "Parliament Class A".

The CSV files from 2022 onward also include access_decision_reasons which contains a pipe-separated list of the unnormalised values.