Embeddings Augmented by Random Permutations (EARP)

This data set includes the word embeddings used for our CoNLL 2018 paper, "Bringing Order to Neural Word Embeddings with Embeddings Augmented By Random Permutations".

As described in the paper, encoding word order with random permutations leads to substantive improvements in performance across multiple analogical retrieval tasks, most notably for "syntactic" analogies that involve mapping between morphological derivatives. Smaller improvements are also evident in downstream sequence labeling tasks. 

Both baseline Skipgram-with-negative-sampling and permutation-based vectors are provided in two: formats - the Semantic Vectors (https://github.com/semanticvectors/semanticvectors) binary format (.bin) and the word2vec binary format (.w2v.bin).

All models were created with two different sliding window configurations - radius 2 (r2) and radius (r5); as well as with and without subword embeddings (sw). The models are as follows:

basic, fastText: standard Skipgram-with-negative-sampling, implemented in Semantic Vectors and fastText respectively

earp_directional: these models use random permutations (EARP) to distinguish between positions before and after the focus term in a sliding window

earp_positional: these models use random permutations (EARP) to distinguish between each position in a sliding window

earp_proximity: these models use random permutations (EARP) to generate encodings that are different, but not orthogonal, for neighboring positions in a sliding window. 

earpx variants: these models use exact sliding window position without replacing subsampled terms, as occurs in other models. 

The word2vec binary editions include random vectors for some characters and terms that were eliminated by Semantic Vectors' Lucene-based tokenization procedure - in particular punctuation marks - as these are useful for certain downstream machine learning tasks.