An Empirical Study of Word Embedding Dimensionality Reduction

In order to analyze the impact on model quality while reducing the number of dimensions, strictly controlled trainings of word embedding are performed on Wikipedia corpora of 170 languages. The specially designed word embedding training tool makes use of processed corpus and intermediate results to accelerate the training, while keeping the consistency of negative sampling.

Tests of semantic relatedness show that, except for some corpora of poor scale, the margin gain from extra dimensions significantly decreases above 200.