Published November 1, 2020 | Version 1
Journal article Open

Bicleaner at WMT 2020: Universitat d'Alacant-Prompsit's submission to the parallel corpus filtering shared task

  • 1. Universitat d'Alacant
  • 2. Prompsit Language Engineering

Description

This paper describes the joint submission of Universitat d’Alacant and Prompsit Language Engineering to the WMT 2020 shared task on parallel corpus filtering. Our submission, based on the free/open-source tool Bicleaner, enhances it with Extremely Randomised Trees and lexical similarity features that account for the frequency of the words in the parallel sentences to determine if two sentences are parallel. To train this classifier we used the clean corpora provided for the task and synthetic noisy parallel sentences. In addition we re-score the output of Bicleaner using character-level language models and n-gram saturation.

Files

2020.wmt-1.107.pdf

Files (289.3 kB)

Name Size Download all
md5:375500b76c742f0b71b65ab55a360058
289.3 kB Preview Download