Preprocessed Penn Tree Bank

Penn Tree Bank is originally a corpus of English sentences with linguistic structure annotations. This is a variant originally distributed by Mikolov here and Zaremba here which omits the annotation and splits the dataset into raining, validation, and test.

The dataset consists of 929k training words, 73k validation words, and 82k test words. As part of the preprocessing, words were lower-cased, numbers were replaced with N, newlines were replaced with <eos>, and all other punctuation was removed. The vocabulary is the most frequent 10k words with the rest of the tokens replaced by an <unk> token.