Linked Data

There is a newer version of the record available.

Published June 30, 2021 | Version 2.0
Dataset Open

Tough Tables: Carefully Evaluating Entity Linking for Tabular Data

Creators

  • 1. University of Milano - Bicocca
  • 2. Bocconi University
  • 3. City, University of London

Description

Tough Tables (2T) is a dataset designed to evaluate table annotation approaches in solving the CEA and CTA tasks.
The dataset is compliant with the data format used in SemTab 2019, and it can be used as an additional dataset without any modification. The target knowledge graph is DBpedia 2016-10.
Check out the 2T GitHub repository for more details about the dataset generation.

New in v2.0: We release the updated version of 2T_WD! The target knowledge graph is Wikidata (online instance) and the dataset complies with the SemTab 2021 data format.

This work is based on the following paper:

Cutrona, V., Bianchi, F., Jimenez-Ruiz, E. and Palmonari, M. (2020). Tough Tables: Carefully Evaluating Entity Linking for Tabular Data. ISWC 2020, LNCS 12507, pp. 1–16.

Note on License: This dataset includes data from the following sources. Refer to each source for license details:
- Wikipedia https://www.wikipedia.org/
- DBpedia https://dbpedia.org/
- Wikidata https://www.wikidata.org/
- SemTab 2019 https://doi.org/10.5281/zenodo.3518539
- GeoDatos https://www.geodatos.net
- The Pudding https://pudding.cool/
- Offices.net https://offices.net/
- DATA.GOV https://www.data.gov/

THIS DATA IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Changelog:

v2.0

  • New GT for 2T_WD
    • A few entities have been removed from the CEA GT, because they are no longer represented in WD (e.g., dbr:Devonté points to wd:Q21155080, which does not exist)
    • Tables codes and values differ from the previous version, because of the random noise.
    • Updated ancestor/descendant hierarchies to evaluate CTA.

v1.0

  • New Wikidata version (2T_WD)
  • Fix header for tables CTRL_DBP_MUS_rock_bands_labels.csv and CTRL_DBP_MUS_rock_bands_labels_NOISE2.csv (column 2 was reported with id 1 in target - NOTE: the affected column has been removed from the SemTab2020 evaluation)
  • Remove duplicated entries in tables
  • Remove rows with wrong values (e.g., the Kazakhstan entity has an empty name "''")
  • Many rows and noised columns are shuffled/changed due to the random noise generator algorithm
  • Remove row "Florida","Floorida","New York, NY" from TOUGH_WEB_MISSP_1000_us_cities.csv (and all its NOISE1 variants)
  • Fix header of tables:
    • CTRL_WIKI_POL_List_of_current_monarchs_of_sovereign_states.csv
    • CTRL_WIKI_POL_List_of_current_monarchs_of_sovereign_states_NOISE2.csv
    • TOUGH_T2D_BUS_29414811_2_4773219892816395776___videogames_developers.csv
    • TOUGH_T2D_BUS_29414811_2_4773219892816395776___videogames_developers_NOISE2.csv

v0.1-pre

  • First submission. It contains only tables, without GT and Targets.

Notes

GT and Target files have different formats in 2T and 2T_WD (CEA). 2T complies with the SemTab2019 CEA format (tab_id, col_id, row_id, entity), while 2T_WD follows the SemTab2020 CEA format (tab_id, row_id, col_id, entity). Visit the SemTab challenge website (https://www.cs.ox.ac.uk/isg/challenges/sem-tab/) for more details.

Files

2T.zip

Files (37.4 MB)

Name Size Download all
md5:ba991395e6b128ae54341dc0fbeecc5f
30.0 MB Preview Download
md5:fc8404db092692b2a4a78522e88bfcb4
7.4 MB Preview Download