Annotation collection Open Access

Asian Directories: Foreign Residents Benchmark Dataset

Cornwell, Peter J; Herren-Oesch, Madeleine

Data manager(s)
Granville, Daniel; Marti, Sorin
Researcher(s)
Futter, Christian; Craig, Matthew J
Work package leader(s)
Decker, Eric L

This record comprises a benchmark dataset based on the listings of foreign residents in the 1896, 1899 and 1934, 1937 volumes of the Asian Directories & Chronicles serial, which was published annually by The Hong Kong Daily Press between 1863 and 1941. With the current exceptions of 1866, 1867, 1872, 1875 and 1884 all of the volumes of the Directories & Chronicles have been assembled by the Europa Institute at the University of Basel. In a collaboration with Data Futures, high-resolution digitization of the pages of the volumes and analysis of OCR data has enabled automated detection of each person record in the foreign resident listings and generation of 60,712 annotations. The latter are represented here as OADM, a precursor of WADM which is currently in widespread use, although this dataset will be upgraded to WADM as soon as more applications emerge which support it. The OCR text has subsequently been corrected and tokenized with the aid of surname and location dictionaries created from the corpus, to produce searchable person 'instance' data using the schema at https://schemata.hasdai.org/historic-persons/historic-person-entry-v0.0.2.json and attached hereto.

The years selected for this benchmark serve the dual purposes of developing dynamic dictionaries for automating correction and tokenization of the remaining volumes of the serial, and they are also pivotal in relation to historic events in East Asia. The First Sino-Japanese War, waged from July 1894 to April 1895, was followed by establishment of large numbers of small communities of foreign residents throughout East Asia. In the early 20th century consolidation of foreign residents in larger communities in coastal cities was followed by a marked exodus during escalating conflict in the Second Sino-Japanese War between July 1937 and September 1945, which some sources date back to the Japanese invasion of Manchuria in 1931. These population shifts are visible when the benchmark dataset is rendered on to maps, as shown in the PDF file attached here. The annotations and instance data are presented at https://fr-benchmark.unibas.hasdai.org/ as an Invenio repository, enabling searching by person and location, as well as viewing person instances in context of the page of the serial where the person was listed—via Mirador and a IIIF service.

 

Swiss National Science Foundation grant 100011_184860/1 "Divisive Power of Citizenship"
Files (240.5 MB)
Name Size
adc_1935202.pdf
md5:1873e7b97912b2ba2ca5a1b4be014676
5.4 MB Download
fr-1896-20191203.json
md5:fedd064141fc5c4f75fd654fd267eff7
13.6 MB Download
fr-1896-ac-20191203.json
md5:87ee34ca4904a2e179fc5f97e2eda2ac
40.4 MB Download
fr-1899-20191203.json
md5:e5d1ba86aa0d0bda946c387617cbc045
15.7 MB Download
fr-1899-ac-20191203.json
md5:15328e5f3c04ff44975982f2e8027343
46.5 MB Download
fr-1934-20191203.json
md5:16081e28f160e07573df5fc08d35d774
19.1 MB Download
fr-1934-ac-20191203.json
md5:5b64449ed97e9c0a504579a4899e907d
54.3 MB Download
fr-1937-20191203.json
md5:675d9849572a3e6cf931ab9defbff89b
11.5 MB Download
fr-1937-ac-20191203.json
md5:6fa69da80909ed2f908fd531937616e9
34.0 MB Download
historic-person-entry-v0.0.2.json
md5:06d29763e7a06ee821954e27bf85f4ee
3.0 kB Download
527
428
views
downloads
All versions This version
Views 527523
Downloads 428420
Data volume 3.2 GB3.1 GB
Unique views 432430
Unique downloads 336332

Share

Cite as