THCHS-30 - Aligned IPA transcriptions

This upload contains aligned IPA transcriptions for the THCHS-30 dataset from OpenSLR. Thereby, punctuation is added, silence marked and duration markers for each phoneme are assigned. Furthermore, the silence on the beginning and ending of each file is marked.

The words were transcribed using pypinyin (v0.47.1) via dict-from-pypinyin (v0.0.1) and mapped to IPA using the pinyin-ipa-map-TONE3-all.json mapping from here. The alignment was done using Montreal Forced Aligner (v2.0.5) and the acoustic model Mandarin MFA (v2.0.0a).

Phoneme duration markers:

• ˘ -> [0, 20) percentile (speaker-wise), e.g., a˥˩˘
• (none) -> [20, 80) percentile (speaker-wise), e.g., a˥˩
• ˑ -> [80, 90) percentile (speaker-wise), e.g., a˥˩ˑ
• ː -> [90, inf) percentile (speaker-wise), e.g., a˥˩ː

Thereby each phoneme (including tones) was considered on its own, i.e., phonemes with different tones were not considered together for the percentile calculation.

Silence markers:

• SILX -> silence at start/end of a recording (aligned on all tiers)
• SIL0 -> no silence
• SIL1 -> [0, 33.33333333) percentile of all silences (speaker-wise)
• SIL2 -> [33.33333333, 66.66666666) percentile of all silences (speaker-wise)
• SIL3 -> [66.66666666, inf) percentile of all silences (speaker-wise)

Files:

• grids.zip
  • contains TextGrids for all audio files containing three tiers words, phonemes and transcription
    • words contains the aligned Chinese words
    • phonemes contains the IPA pronunciations including silence markers at start and end (SILX)
    • transcription contains unaligned phonemes including punctuation and word boundary labels (SIL0)
  • the folder structure is equal to one from the dataset
• grids-sdp.zip
  • same as grids.zip except the folder structure is the one from speech-dataset-parser (v0.0.4)
• preview-start/middle/end.png
  • preview of the first TextGrid from speaker A2 opened in Praat in different positions
• words-vocabulary.txt
  • contains all Chinese words from tier words
• phonemes-vocabulary.txt
  • contains all phonemes from tier phonemes
• transcription-vocabulary.txt
  • contains all phonemes/punctuation from tier transcription
• phonemes-durations.pdf
  • contains the plotted phoneme duration distribution of tier phonemes
• phonemes-durations-simple.pdf
  • contains the plotted phoneme duration distribution of tier phonemes if all duration markers are ignored
• phonemes-durations-simple-toneless.pdf
  • contains the plotted phoneme duration distribution of tier phonemes if all duration markers and tones are ignored
• pronunciations-broad.dict
  • contains the broad pronunciations for each word including punctuation but not duration markers
  • e.g., 一下。 i˥ ɕ j a˥˩ 。
• pronunciations-narrow.dict
  • contains the narrow pronunciations for each word including punctuation, duration markers and weights (= occurrence) over all speakers
  • e.g., 一下。 3 i˥ ɕ j a˥˩ː 。
• pronunciations-narrow-speakers.zip
  • contains the narrow pronunciations separated for each speaker
• script.sh
  • contains the script to reproduce all results
  • error in line 32: replace with speech-dataset-parser==0.0.4